Opensuse Leap, X, xauth, flatpak vs. kate – deviating selection of magic cookie entries in .Xauthority – I

Posted on 12. January 2023 by eremo

On an average Linux system xorg’s X services or (X)Wayland services are fundamental building blocks for graphical user interfaces like KDE. We use these services every day without thinking much about security measures and other processes in the background. But sometimes strange events wake you up from daily routine. This blog post got triggered by such an event – namely an odd and varying reaction of an X-client application to changes of the network name of a Linux host (running Opensuse Leap 15.3).

This post focuses on a standard cookie-based access mechanism of X-clients to local Xorg-services on the very same Linux host. The basic idea is that the X-server grants access to its socket and services only if the X-client presents a fitting “secret” cookie. The required magic cookie value is defined by the X-server when it is started. The X-client gets information about X-related cookies from a user specific file ~/.Xauthority which may contain multiple entries. I will show that changing of the name of the host, i.e. the hostname, may have an unexpected impact on the subsequent start of X-clients and that different X-clients may react in deviating ways to a hostname change.

The ambiguity in the startup process of an X-client after a hostname change – even if the correct cookie is in principle available – is demonstrated by different reactions of a standard KDE application, namely kate, and a flatpak-based application, namely Blender. Flatpak is of interest as there are many complaints on the Internet regarding an unexplained denial of X-server access, especially on Opensuse systems.

We first look a bit into basic settings for a standard MIT-1 cookie based access mechanism. (For an in-depth introduction to X-server security readers will have to consult specific documentation available from Xorg). Afterward I will show that the rules for selecting a relevant cookie entry in the file .Xauthority deviate between the applications named above. The analysis will show that there is an intimate relation of these rules with available sources of the hostname and .Xauthority entries for various hosts. Hostname changes during running X-sessions may, therefore, have an impact on the start of graphical applications. An unexpected negative impact may even occur after a restart of the DisplayManager.

In this post we look at the effects of hostname changes during a running X-session, e.g. a KDE-session. In further posts I will look a bit deeper into sequences of actions a user may perform following a hostname change – such as a simple logout from and a new login into a graphical session or a full restart of systemd’s graphical target. Even then the consequences of a hostname change may create confusion. If and when I find the time I may also look into aspects of Wayland access in combination with KDE applications and flatpak.

What had happened? Denial and acceptance of Xorg-server access for a flatpak-based application after changing the host’s name …

Some days ago, I came to a location where I had to use a friend’s WLAN router to get an Internet connection. On my laptop NetworkManager controls the WLAN access. NetworkManager has an option to submit a hostname to the router. As I do not want my standard hostname to be spread around I first changed it via YaST during a running KDE-session. Then I configured NetworkManager for the local WLAN and restarted the network. Access to the WLAN router and Internet worked as expected. However, when I tried to start a flatpak based Blender installation I got the message

Invalid MIT-MAGIC-COOKIE-1 key Unable to open a display

This appeared to be strange because the flatpak Blender installation had worked flawlessly before the change of the hostname and before WLAN access. The question that hits you as a normal Linux user is: What do a hostname change and a network restart have to do with the start of a flatpak application in an already running KDE session?

Then I changed the hostname again to a different string – and could afterward start flatpak Blender without any X-authorization problem. Confused? I was.

Then I combined the change of the hostname with an intermediate start of the display manager and/or a stop and restart of systemd’s “graphical target” – and got more disturbing results. So I thought: This is an interesting area worth of looking into it a bit deeper.

The error message, of course, indicated problems with an access to the display offered by the running X-server. Therefore, I wanted answers to the following questions:

Why did the magic cookie fail during an already running X-session? Had the X-access conditions not been handled properly already when I logged myself into my KDE session?
Why was the magic cookie information invalid the first time, but not during the second trial?
What impact has the (changed) hostname on the cookie-based X-authorization mechanism?
What can I check by which tool regarding xauth-problems?
What does flatpak exactly send to the X server when asking for access?
Which rules govern the cookie based X-authorization mechanism for other applications than flatpak-based applications?

I had always assumed that X socket access would follow clear rules – independent of a specific X-client. But after some simple initial tests I started wondering whether the observed variations regarding X access had something to do with entries in the ~/.Xauthority file of the user which I had used to log into my KDE-session. And I also wondered whether a standard KDE application like “kate” would react to hostname changes in really the same way as a flatpak application.

Changes of ~/.Xauthority entries …

During a series of experiments I started to manipulate the contents of the ~/.Xauthority file of the user of a running KDE session. Whilst doing so I compared the reaction of a typical KDE application like kate with the behavior of a (sandboxed) flatpak Blender installation. And guess what: The reactions were indeed different.

You find the details of the experiments below. I have not yet analyzed the results with respect to potential security issues. But the variability in the selection and usage of .Xauthority entries by different applications appears a bit worrisome to me. In any case the deviating reactions at least have an impact upon whether an X-client application does start or does not start after one or multiple hostname changes. This makes an automatic handling of certain X-client starts by user scripts a bit more difficult than expected.

“Transient” state of a Linux system regarding its hostname?

The basic problem is that a user can create a transient system status regarding a change of the static name of a host: A respective entry in the file /etc/hostname may differ from values which other resources of the running graphical desktop session may provide to the user or programs. Transient is a somewhat problematic term. E.g. the command “hostnamectl” may show a transient hostname for some seconds when you change the contents of /etc/hostname directly – until a (systemd) background process settles the status. So what exactly do I mean by “transient”?

By “transient” I mean that a change of the static hostname in /etc/hostname may not be reflected by environment variables containing hostname information, e.g. by the variables HOST and HOSTNAME. Conflicting hostname information may occur as long as we do not restart the display manager and/or restart the “graphical target” of systemd after a change of the hostname.

To my astonishment I had to learn that it is not self-evident what will happen in such transient situations regarding access authorization to an Xorg-server via a so called “magic MIT-1 cookie“. This is partially due to the fact that the file “~/.Xauthority” may contain multiple entries. See below.

Objectives of this post

In this first post on the topics named above I will have a look at what happens

when we change the static hostname during a running X-session (e.g. a KDE session),
when we change entries in the ~/.Xauthority file during a running X-session, in particular with respect to hostname(s), and try to start an X-client afterwards.

The second point will help us to identify rules which X-client applications follow whilst choosing and picking up a cookie value among multiple entries in the file “~/.Xauthority”. We compare the behavior of “kate” as an example for a KDE application with the reaction of a sandboxed flatpak application (Blender). The astonishing result will be that the rules really differ. The rules for flatpak may prevent a start of a flatpak application although a valid entry may be present in .Xauthority.

Our starting point: A consistent system-status regarding the hostname in a LAN or WLAN environment

The whole problem of an invalid cookie started with a hostname change ahead of a WLAN access. Why is a defined hostname important? How do we change it on an Opensuse system? What are potential sources on the system regarding information about the present hostname? And what should a consistent situation at the beginning of experiments with hostname changes look like?

In a typical structured LAN/WLAN there are multiple hosts wanting to interact with servers on the Internet, but also with each other or servers in (sub-) networks. As an administrator you may define separate sub-nets behind a common gateway. Members of a sub-net may be objects to routing and firewall restrictions regarding the communication with other hosts in the same or other sub-nets or servers on the Internet. As we humans operate with host-names rather than IP-addresses we may give hosts unique names in a defined (sub-) network, use DHCP to statically or dynamically assign IP addresses (and maybe even host-names) and use DNS-services to translate hostnames and related FQDNs into IP-addresses. In a local LAN/WLAN you may have full control over all these ingredients and design a consistent landscape of named and interacting hosts. Plus a pre-designed network segregation, host- routers, gateways, gateway-/perimeter-firewalls, etc.

The situation may be different when you put your laptop in a foreign and potentially dangerous WLAN environment. In my case I use Networkmanager on Leap 15.3 (soon 15.4) to configure WLAN access. The WLAN routers in most networks I have to deal with most often offer both DHCP and DNS services. The IP-address is assigned dynamically. The router’s DNS server works as a forwarder with respect to the Internet. But regarding the local network, which the router controls, the DNS-service of the router may respect your wishes regarding a hostname or not. And you do not know what other hosts creep around in the network you become a member of – and how they see you with respect to your host’s name.

There are three important things I want to achieve in such a situation:

As soon as a WLAN connection to a router gets up I want to establish firewall rules blocking all incoming traffic and limit the outgoing traffic as much as possible.
I do not want the WLAN routers to mingle with the hostname set by me, because this name play a role in certain scripts and some host-internal virtual network environments.
Other friendly hosts in the network may ping me under a certain defined hostname – which the DNS part of the WLAN router should be informed about.

All these things can be managed directly or indirectly by Networkmanager (and some additional scripts). In particular you can a start a script that installs netfilter rules for a certain pre-defined hostname and does further things – e.g. delete or supplement entries in “/etc/hosts”.

However, the host’s name has somehow to be defined ahead of the network connection. There are multiple options to do this. One is to edit the file “/etc/hostnames”. Another is offered by YaST on Opensuse systems. A third is a script of your own, which in addition may manage settings in /etc/hosts, e.g. regarding virtual networks controlled by yourself.

Under perfect circumstances you may achieve a status where you have a well defined static hostname not touched by the WLAN router, a local host-based firewall controlling all required connections and your chosen hostname has been accepted by the router and integrated into its own DNS domain.

Let us check this for the WLAN environment at one of my friends locations:

myself@xtux:~> hostname; cat /etc/hostname; echo $HOST; echo $HOSTNAME; echo $XAUTHLOCALHOSTNAME; echo $SESSION_MANAGER; hostnamectl
xtux
xtux
xtux
xtux
xtux
local/xtux:@/tmp/.ICE-unix/7289,unix/xtux:/tmp/.ICE-unix/7289
   Static hostname: xtux
         Icon name: computer-laptop
           Chassis: laptop
        Machine ID: ...... (COMMENT: long ID) 
           Boot ID: b5d... (COMMENT: long ID)
  Operating System: openSUSE Leap 15.3
       CPE OS Name: cpe:/o:opensuse:leap:15.3
            Kernel: Linux 5.3.18-150300.59.101-default
      Architecture: x86-64
myself@xtux:~> 

myself@xtux:~ # ping xtux
PING xtux.home (192.168.10.186) 56(84) bytes of data.
64 bytes from xtux.home (192.168.10.196): icmp_seq=1 ttl=64 time=0.030 ms
64 bytes from xtux.home (192.168.10.196): icmp_seq=2 ttl=64 time=0.060 ms
^C
--- xtux.home ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2013ms

(The COMMENTs are not real output, but were added by me.)
You see that there is a whole variety of potential sources regarding information about the host’s name. In particular there are multiple environment variables. In the situation shown above all information sources agree about the host’s name, namely “xtux”. But you also see that the local domain name used by the WLAN router is a default one used by a certain router vendor. You would get the domain present name used also by issuing the command “dnsdomainname” at a shell-prompt.

An interesting question is: Which sources will register a hostname change during a running KDE session? Another interesting question is: Is the hostname used for local X-access authorization? The answers will be given by the experiments described below.

Cookie based X-server access by the MIT-MAGIC-COOKIE-1 mechanism

X-server access can be controlled by a variety of mechanisms. The one we focus on here is a cookie-based access. The theory is that an X-server when it starts up queries the hostname and creates a “secret” cookie plus a hash defining a file where the server saves the cookie. Afterward any X-client must provide this specific magic cookie when trying to get access to the X-server (more precisely to its socket). See e.g. a Wikipedia article for these basic principles.

Let us check what happens when we start a X-server. On an Opensuse system a variety of systemd-services is associated with a pseudo-state “3” of the host which can be set by a command “init 3”. This “state” corresponds, of course, to a systemd target. Network and multiuser operations are provided. A command “init 5” then moves the system to the graphical target. This includes the start of the Display Manager – in my case sddm. sddm in turn starts xorg’s X server. On an Opensuse system see the file /etc/sysconfig/displaymanager for respective settings.

Now let us get some information from the system about these points. First we look at the command which had been used to start the X-server. As root:

xtux:~ # pgrep -a X
16690 /usr/bin/X -nolisten tcp -auth /run/sddm/{0ca1db02-e253-4f2b-972f-9b124764a65f} -background none -noreset -displayfd 18 -seat seat0 vt7

The path after the -auth option gives us the location of the file containing the “magic cookie”. We can analyze its contents by the xauth command:

xtux:~ # xauth -f /run/sddm/\{0ca1db02-e253-4f2b-972f-9b124764a65f\} list
xtux/unix:0  MIT-MAGIC-COOKIE-1  650ade473bc07c2e981d6174871c2ad0

(Hint: The tab-key will help you to avoid retyping the long hash). Ok, here we have our secret cookie for further authentication of applications wanting to get access to the X-server’s socket.

The other side of the story is the present user who opened an X-session – in my case a KDE session. Where does it or any X-clients, which the user later starts on a graphical desktop, get the knowledge about the required cookie from?

Answer: Whilst logging in via the DisplayManager an entry is written into or replaced inside the file ~/.Xauthority (by a root controlled process). If this file does not exist it is created.

The .Xauthority file should normally not be edited as it contains non-interpretable characters. But the command “xauth list” helps again to present the contents in readable form. It automatically picks the file ~/.Xauthority:

myself@xtux:~> xauth list
xtux/unix:0  MIT-MAGIC-COOKIE-1  650ade473bc07c2e981d6174871c2ad0

We see the same cookie here as on the local X-server’s side. Any applications with graphical output can and should use this information to get access to the X-server. E.g. “ssh -X” can use this information to rewrite cookie information set during interaction with a remote system to the present local X-server cookie. And there are of course the sandboxed flatpak applications in their namespaces. Note that the screen used by definition is :0 on the local host.

Note further that the situation regarding .Xauthority is ideal: In our present situation it contains only one entry. On systems working a lot on different remote hosts this file normally contains multiple entries, in particular if you have opened X-connections to other hosts in the past. Or, if you have changed your hostname before …

Access to the running X-server by a flatpak-based Blender installation

When we start a flatpak Blender installation this will just open the Blender interface on the hosts graphical desktop screen. We can close Blender directly afterward again.

myself@xtux:~> flatpak run org.blender.Blender &
[1] 23145
myself@xtux:~> Saved session recovery to '/tmp/quit.blend'

Blender quit

[1]+  Fertig                  flatpak run org.blender.Blender
myself@xtux:~>

We shall later see that flatpak replaces a screen-number :99 used in the applications namespace to :0 when interacting with the locally running X-server. In this respect it seems to work similar to “ssh -X”.

Disturbing the hostname settings

What happens if we disturb the initial consistent setup by changing the host’s name during a running X-session?

We have already identified potential sources of a mismatch regarding the hostname: /etc/hostname vs. a variety of environment variables vs. the DNS system.
The reader has certainly also noticed that the entry in the ~/.Xauthority file starts with the present name of the host. So here we have yet another potential source of a mismatch after a change of the hostname.

There are obvious limitations: The present cookie value stored at the location /run/sddm/\{..} should not be overwritten. This may disturb running X-clients and the start of other application windows in sub-shells of the running KDE-session. Changing environment variables which contain the hostname, may be dangerous, too.

On the other side the entry in ~/.Xauthority may not fit the new hostname if the entry is not being adapted. Actually, changing the hostname via YaST on an Opensuse system leaves .Xauthority entries as they were. So how would a KDE application and a flatpak application react to such discrepancies?

Basic Experiment: Change the hostname during a KDE session and try to run a flatpak-based Blender application afterward

There are multiple options to change the hostname. We could overwrite the contents of /etc/hostname directly as root. (And wait for some systemd action to note the difference) But that does not automatically change other network settings. Let us use therefore use YaST in the form of the graphical yast2.

We just ignore the warning and define a new hostname “xmux”. Then we let yast2 do its job to reconfigure the network settings. Note that this will not interrupt an already running NetworkManager-controlled WLAN connection. Afterward we check our bash environment:

myself@xtux:~> hostname; cat /etc/hostname; echo $HOST; echo $HOSTNAME; echo $XAUTHLOCALHOSTNAME; echo $SESSION_MANAGER; hostnamectl
xmux
xmux
xtux
xtux
xtux
local/xtux:@/tmp/.ICE-unix/7289,unix/xtux:/tmp/.ICE-unix/7289
   Static hostname: xmux
         Icon name: computer-laptop
           Chassis: laptop
        Machine ID: .....   (COMMENT: unchangd long string) 
           Boot ID: 65c..   (COMMENT: changed long string)
  Operating System: openSUSE Leap 15.3
       CPE OS Name: cpe:/o:opensuse:leap:15.3
            Kernel: Linux 5.3.18-150300.59.101-default
      Architecture: x86-64
myself@xtux:~> xauth list
xtux/unix:0  MIT-MAGIC-COOKIE-1  650ade473bc07c2e981d6174871c2ad0

We got a new bootid, but neither the environment variables nor the contents of ~/.Xauthority have been changed. Will our flatpak-based Blender run?
Answer: Yes, it will – without any warnings! Will kate run? Yes, it will, but with some warning:

myself@xtux:~> kate &
[1] 6685
myself@xtux:~> No protocol specified

The reaction of kate is a bit questionable. Obviously, it detects an unexpected discrepancy, but starts nevertheless.

There are 2 possible explanations regarding flatpak: 1) flatpack ignores the hostname in the .Xauthority entry. It just reacts to the screen number. 2) flatpak remembers the last successful cookie and uses it.

How can we test this?

Preparation of further experiments: How to change the contents of .Xauthority manually

One way to test the reaction of applications to discrepancies of a changed hostname with cookie entries in .Xauthority is to manually change entries in this file. This is not as easy as it may seem as this file has weird characters in it. But it is possible with the help of kate or kwrite. You need an editor which can grasp most of the symbols in a sufficient way. vi is not adequate.

BUT: You have to be very careful when copying entry lines. Identify the symbol sequence marking the beginning of an entry is a first important step. Note also: When you change a hostname of an entry you must use one of the same length!
AND: Keep a copy of the original .Xauthority-file somewhere safe during your experiments.

Multiple entries in ~/.Xauthority

The file ~/.Xauthority can in principle have multiple entries for different hosts and different X-servers. Among other scenarios multiple entries are required to cover situations for

for direct access of a local X-client via a network to other hosts’ X-servers
for remote application access based on “ssh -X” and a display of the output on the local host’s X-server.

Furthermore: A restart of an X-server will lead to new additional entries in ~/.Xauthority, whilst existing entries are kept there.

Therefore, it is wise to work with multiple entry lines in ~/.Xauthority for further experiments. However: Multiple entries with a combination of hostnames and cookie values open up a new degree of freedom:

The decision whether an X-client application gets successfully started or not will not only depend on a cookie match at the X-server, but also on the selection of an entry in the file ~/.Xauthority.

Hopefully, the results of experiments, which mix old, changed and freely chosen hostnames with valid and invalid cookie values will give us answers to the question of how a hostname change affects running X-sessions and freshly started X-clients. We are especially interested in the rules that guide an application when it must select a particular entry in .Xauthority among others. If we are lucky we will also get an idea about how an intermediate restart of the X-server after a hostname change may influence the start of X-clients afterward.

During the following experiments I try to formulate and improve rules regarding kate and flatpak applications with respect to the selection and usage of entries in .Xauthority.

Please note:

In all forthcoming experiments we only consider local applications which try to gain access to a locally running X-server!

The cookie of an entry in ~/.Xauthority is considered to be “correct” if it matches the magic cookie of the running X-server. Otherwise we regard it as “incorrect” with respect to the running X-server.

Experiment 1: Change of the file ~/.Xauthority – leading entry for the original hostname with an incorrect cookie, second entry for new hostname with the correct cookie

I prepared a file /home/myself/.Xauthority (as root) during the running X-session with the following entries:

myself@xtux:~> xauth list
xtux/unix:0  MIT-MAGIC-COOKIE-1  650ade473bc07c2e981d6174871c2a44
xmux/unix:0  MIT-MAGIC-COOKIE-1  650ade473bc07c2e981d6174871c2ad0
myself@xtux:~> kate
myself@xtux:~> flatpak run org.blender.Blender &
[1] 28187
myself@xtux:~> Invalid MIT-MAGIC-COOKIE-1 keyUnable to open a display
^C
[1]+  Exit 134                flatpak run org.blender.Blender
myself@xtux:~>

You see that I have changed the hostname (xtux) of our original entry (with the correct cookie) to the meanwhile changed hostname (xmux). This entry is associated with the presently valid magic cookie. And I have added a leading entry with the original hostname, but a modified and therefore invalid cookie value.

Now, let us check what a KDE application like kate would do afterward: The output above shows that it just started – without any warning. However, flatpak did and does NOT start:

myself@xtux:~> flatpak run org.blender.Blender &[1] 16686
myself@xtux:~> Invalid MIT-MAGIC-COOKIE-1 keyUnable to open a display

Experiment 2: ~/.Xauthority with a first entry for the new hostname but with an invalid cookie, plus a second entry for the original hostname with the correct cookie

Let us change the entries to:

myself@xtux:~> xauth list
xmux/unix:0  MIT-MAGIC-COOKIE-1  650ade473bc07c2e981d6174871c2a44
xtux/unix:0  MIT-MAGIC-COOKIE-1  650ade473bc07c2e981d6174871c2ad0
myself@xtux:~> kate
Invalid MIT-MAGIC-COOKIE-1 keymyself@xtux:~> flatpak run org.blender.Blender &
[1] 28371
myself@xtux:~> Invalid MIT-MAGIC-COOKIE-1 keyUnable to open a display
^C
[1]+  Exit 134                flatpak run org.blender.Blender
myself@xtux:~>

It may seem that a line break is missing in the output and that kate did not start. But this is wrong:

Kate actually DID start! But it produced an alarming warning!
However flatpak did NOT start AND gave us a warning!

Meaning:

We got a clear indication that

different entries are used by our two applications and
that the potential discrepancies of the hostnames associated with the .Xauthority-entries in comparison to environment variables and /etc/hostname are handled differently by our two applications.

Why kate starts despite the clear warning is a question others have to answer. I see no direct security issue, but I have not really thought it through.

Experiment 3: ~/.Xauthority with a leading entry for original hostname and the correct cookie plus entry for the new hostname but an incorrect cookie

Let us now change the order of the cookie entries:

myself@xtux:~> xauth list
xtux/unix:0  MIT-MAGIC-COOKIE-1  650ade473bc07c2e981d6174871c2ad0
xmux/unix:0  MIT-MAGIC-COOKIE-1  650ade473bc07c2e981d6174871c2a44
myself@xtux:~> kate
Invalid MIT-MAGIC-COOKIE-1 keymyself@xtux:~> flatpak run org.blender.Blender &
[1] 28571
myself@xtux:~> /run/user/1004/gvfs/ non-existent directory
Saved session recovery to '/tmp/quit.blend'

Blender quit

[1]+  Fertig                  flatpak run org.blender.Blender
myself@xtux:~>

kate again gives us a warning, but starts.
And, oh wonder, flatpak now does start Blender in its namespace without any warning!

Experiment 4: ~/.Xauthority with a leading entry for the new hostname and the correct cookie, plus an entry for the original hostname but an invalid cookie

Let us switch the hostnames again in the given order of the cookie entries. This gives us the last variation for mixing the new and the old hostnames with valid/invalid cookies :

myself@xtux:~> xauth list
xmux/unix:0  MIT-MAGIC-COOKIE-1  650ade473bc07c2e981d6174871c2ad0
xtux/unix:0  MIT-MAGIC-COOKIE-1  650ade473bc07c2e981d6174871c2a44
myself@xtux:~> kate
myself@xtux:~> flatpak run org.blender.Blender &
[1] 28936
myself@xtux:~> /run/user/1004/gvfs/ non-existent directory
Saved session recovery to '/tmp/quit.blend'

Blender quit

[1]+  Fertig                  flatpak run org.blender.Blender
myself@xtux:~>

kate now starts without any warning. And also flatpak starts without warning.

Intermediate, but insufficient interpretation – reason for further experiments

How can we interpret the results above? So far the results are consistent with the following rules:
kate: Whenever the cookie associated with the present static hostname in .Xauthority matches the X-server’s cookie kate will start without warning. Otherwise it issues a warning, but starts nevertheless.
flatpak: Whenever the first entry in .Xauthority provides a cookie that matches the X-server’s cookie for the session than flatpak starts an X-client program like Blender.

But things are more complicated than this. We also have to check what happens if we, for some reason, have entries in ~/.Xauthority that reflect a hostname neither present in /etc/hostname nor in environment variables. (Such an entry may have resulted from previous access trials to the X-servers of remote hosts or “ssh -X” connections.)

I will call such a hostname a locally “unknown hostname” below. It admit that this is not the best wording, but it is a short one. A “known hostname” instead is one either provided by environment variables or being present in /etc/hostname.

Experiment 5: ~/.Xauthority with a leading entry for an unknown hostname and the correct cookie, plus an entry for the new hostname but an incorrect cookie

Entries of the form

myself@xtux:~> xauth list
xfux/unix:0  MIT-MAGIC-COOKIE-1  650ade473bc07c2e981d6174871c2ad0
xmux/unix:0  MIT-MAGIC-COOKIE-1  650ade473bc07c2e981d6174871c2a44

reflect such a situation.

The reactions of both kate and flatpak are negative in the given situation:

myself@xtux:~> xauth list
xfux/unix:0  MIT-MAGIC-COOKIE-1  650ade473bc07c2e981d6174871c2ad0
xmux/unix:0  MIT-MAGIC-COOKIE-1  650ade473bc07c2e981d6174871c2a44
myself@xtux:~> kate
Invalid MIT-MAGIC-COOKIE-1 keyNo protocol specified
qt.qpa.xcb: could not connect to display :0
qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found.
Failed to create wl_display (No such file or directory)
qt.qpa.plugin: Could not load the Qt platform plugin "wayland" in "" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Available platform plugins are: wayland-org.kde.kwin.qpa, eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, wayland-egl, wayland, wayland-xcomposite-egl, wayland-xcomposite-glx, xcb.

Abgebrochen (Speicherabzug geschrieben)
myself@xtux:~> 
myself@xtux:~> flatpak run org.blender.Blender &
[1] 30565
myself@xtux:~> Invalid MIT-MAGIC-COOKIE-1 keyUnable to open a display
^C
[1]+  Exit 134                flatpak run org.blender.Blender
myself@xtux:~>

Meaning:

flatpak reacts allergic to entries with unknown hostnames and to entries with known hostnames, but with a wrong cookie.

Experiment 6: ~/.Xauthority with a leading entry for the new hostname and the correct cookie, plus an entry for an unknown hostname with an invalid cookie

However:

myself@xtux:~> xauth list
xmux/unix:0  MIT-MAGIC-COOKIE-1  650ade473bc07c2e981d6174871c2ad0
xfux/unix:0  MIT-MAGIC-COOKIE-1  650ade473bc07c2e981d6174871c2a44
myself@xtux:~> kate
myself@xtux:~> flatpak run org.blender.Blender &
[1] 30710
myself@xtux:~> /run/user/1004/gvfs/ non-existent directory
Saved session recovery to '/tmp/quit.blend'

Blender quit

[1]+  Fertig                  flatpak run org.blender.Blender
myself@xtux:~>

Both applications start without warning!

Experiment 7: ~/.Xauthority with a leading entry for the original hostname and the correct cookie, plus an entry for an unknown hostname with an invalid cookie

The reaction of our applications changes again for the following settings:

myself@xtux:~> xauth list
xtux/unix:0  MIT-MAGIC-COOKIE-1  650ade473bc07c2e981d6174871c2ad0
xfux/unix:0  MIT-MAGIC-COOKIE-1  650ade473bc07c2e981d6174871c2a44
myself@xtux:~> kate
No protocol specified
myself@xtux:~> flatpak run org.blender.Blender &
[1] 30859
myself@xtux:~> /run/user/1004/gvfs/ non-existent directory
Saved session recovery to '/tmp/quit.blend'

Blender quit

[1]+  Fertig                  flatpak run org.blender.Blender
myself@xtux:~>

Meaning:

If the hostname associated with the right cookie is present in the environment variables, but does not correspond to the contents of /etc/hostname then kate will start with some warning. Flatpak starts starts without warning.

Experiment 8: ~/.Xauthority with a leading entry for an unknown hostname and an incorrect cookie, plus an entry for a known hostname with an invalid cookie

Switching entries and renaming confirms previous results:

myself@xtux:~> xauth list
xfux/unix:0  MIT-MAGIC-COOKIE-1  650ade473bc07c2e981d6174871c2a44
xtux/unix:0  MIT-MAGIC-COOKIE-1  650ade473bc07c2e981d6174871c2ad0
myself@xtux:~> kate
No protocol specified
myself@xtux:~> flatpak run org.blender.Blender &
[1] 31113
myself@xtux:~> /run/user/1004/gvfs/ non-existent directory
Saved session recovery to '/tmp/quit.blend'

Blender quit

[1]+  Fertig                  flatpak run org.blender.Blender

And:

myself@xtux:~> xauth list
xfux/unix:0  MIT-MAGIC-COOKIE-1  650ade473bc07c2e981d6174871c2a44
xmux/unix:0  MIT-MAGIC-COOKIE-1  650ade473bc07c2e981d6174871c2ad0
myself@xtux:~> kate
myself@xtux:~> flatpak run org.blender.Blender &
[1] 31348
myself@xtux:~> /run/user/1004/gvfs/ non-existent directory
Saved session recovery to '/tmp/quit.blend'

Blender quit

Experiment 9: ~/.Xauthority with entries for unknown hostnames, only

Let us use unknown hostnames, only.

myself@xtux:~> xauth list
xfuxi/unix:0  MIT-MAGIC-COOKIE-1  650ade473bc07c2e981d6174871c2ad0
xruxi/unix:0  MIT-MAGIC-COOKIE-1  650ade473bc07c2e981d6174871c2ad0
myself@xtux:~> kate
No protocol specified
No protocol specified
qt.qpa.xcb: could not connect to display :0
qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found.
Failed to create wl_display (No such file or directory)
qt.qpa.plugin: Could not load the Qt platform plugin "wayland" in "" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Available platform plugins are: wayland-org.kde.kwin.qpa, eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, wayland-egl, wayland, wayland-xcomposite-egl, wayland-xcomposite-glx, xcb.

Abgebrochen (Speicherabzug geschrieben)
myself@xtux:~> flatpak run org.blender.Blender &
[1] 29115
myself@xtux:~> No protocol specified
Unable to open a display
^C
[1]+  Exit 134                flatpak run org.blender.Blender

So, having unknown hostnames only, will lead to no X-access, neither for flatpak nor kate.

Derived rules for the selection of an entry in ~/.Xauthority

So the rules for our two selected applications regarding the selection of an entry in ~/.Xauthority and resulting X-server access are more like described below:

kate: If an entry in .Xauthority has a known hostname and fits the X server’s cookie kate is started – with a warning, if the hostname does not fit the present static hostname (in /etc/hostname). If there is an additional deviating entry for the present static hostname, but with an incorrect cookie then the warning includes the fact that the cookie is invalid, but kate starts nevertheless.
flatpak Blender: The first entry which matches a known hostname (among the available ones from environment variables or from /etc/hosts) and which matches screen :0 is used to pick the respective cookie for X-server access. The application (X-client) only starts if the resulting cookie matches the X-server’s cookie.
Both: If ~/.Xauthority does not contain entries which match any of the known hostnames than both programs fail regarding X-access. kate does check for other possible sockets (e.g. for (X)Wayland) in this case.

When evaluating these rules with respect to security issues one should always keep in mind that .Xauthority-entries like the ones we have artificially constructed may have been the result of a sequence of hostname changes followed by restarts of the X-server. This will become clearer in the next post.

Conclusion

By some simple experiments one can show that the access to an Xorg-server requested by a X-client application does not only depend on a cookie match but also on the combination of hostnames and associated magic cookie values offered by multiple entries in the file ~/.Xauthority. The rules by which an X-client application selects a specific entry may depend on the application and the rules may differ from those other applications follow. We have seen that at least a flatpak based Blender installation follows other rules than e.g. KDE’s kate. Therefore, changes of the hostname during a running X-session may have an impact on the startup of applications. E.g. if .Xauthority already contains an entry for the new hostname.

The attentive reader has, of course, noticed that the experiments described above alone do not explain the disturbing reaction of flatpak to hostname changes described in the beginning. These reactions had to do with already existing entries for certain hostnames in .Xauthority. Additional entries may in general be the result of previous (successful) accesses to remote hosts’ X-servers or previous local hostname changes followed by X-server restarts. In the next post I will, therefore, extend the experiments to intermediate starts of both the X-server and the graphical target after hostname changes.

Links

Opensuse Leap 15.3 documentation on xauth

Stackoverflow question on “How does X11 authorization work? (MIT Magic Cookie)”

Stackexchange question on Invalid MIT-MAGIC-COOKIE-1 keyxhost: unable to open display “:0”

Opensuse bug Bug 1137491 – flatpak/snap: “Invalid MIT-MAGIC-COOKIE-1 key” after resume (network/hostname changes?)

Also see comment of the 25th og January, 2019, in an archived Opensuse.org contribution

And before we who love democracy and freedom forget it:
The worst fascist, war criminal and killer living today is the Putler. He must be isolated at all levels, be denazified and sooner than later be imprisoned. Somebody who orders the systematic destruction of civilian infrastructure must be fought and defeated because he is a danger to mankind and principles of humanity. Long live a free and democratic Ukraine!

Manual requests to a https web-server with “openssl s_client”: do not forget the options “-ign_eof -crlf”

Posted on 29. December 2020 by eremo

Sometimes you may need to analyze the behavior and responses of a web-server or a REST service to certain requests. And sometimes you are restricted to the command line of a Linux system (e.g. during penetration testing). Then you have to type and send HTTP commands in a direct manner. While this is trivial with telnet and HTTP-commands via n unencrypted connection on port 80, you must use a tool like openssl for HTTPS-servers using TLS tunnels.

A quick search on the Internet will show you that you should be able to use “openssl” on a Linux system in the following form:

openssl s_client -connect YOUR_TARGET_WEB_DOMAIN:443

Or – if you do not want to look at certificates and related CA chains in detail – with an additional option “-quiet”:

openssl s_client -quiet -connect YOUR_TARGET_WEB_DOMAIN:443

YOUR_TARGET_WEB_DOMAIN has to be replaced, of course, by a valid URI. For restricting the encryption to TLS V1.2 you would instead use:

openssl s_client -quiet -tls1_2 -connect YOUR_TARGET_WEB_DOMAIN:443

For some servers an additional option “-ign_eof” can be helpful: This hinders a connection to directly close when an “end of file” [EOF] may be reached (during a response). Meaning: The response will not be shown in some cases. The option “-quiet” triggers a “-ign_eof” behavior implicitly. But keep in mind that this option does not hinder any timeouts on the connection imposed by the server.

A problem with the interactive “openssl s_client” command-line on Linux systems

After entering the above commands at the command prompt of a Linux shell (e.g. bash) you will first see some information regarding the connection handshake and the establishment of the encryption tunnel. Then end up on a line where you can interactively enter HTTP commands. You expect to successfully enter commands in the following way:

We type “GET / HTTP/1.1.” (without the quotes) => we press the “ENTER”-key => we get a new line => we type “Host: YOUR_TARGET_WEB_DOMAIN” (without the quotes) => we press the “ENTER”-key twice.

You may try this sequence with “google.com”. This will work! You get, however, an information that the document has been moved to “www.google.com”. But at the new address everything is working, too.

So far so good! But let us try the given recipe with another domain: www.debian.org. Using the command line within “s_client” then leads to an error:

.....
    Start Time: 1609195616
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no
    Max Early Data: 0
---
read R BLOCK
GET / HTTP/1.1
HTTP/1.1 400 Bad Request
Date: Mon, 28 Dec 2020 22:47:04 GMT
Server: Apache
X-Content-Type-Options: nosniff
X-Frame-Options: sameorigin
Referrer-Policy: no-referrer
X-Xss-Protection: 1
Strict-Transport-Security: max-age=15552000
Content-Length: 291
Connection: close
Content-Type: text/html; charset=iso-8859-1

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>400 Bad Request</title>
</head><body>
<h1>Bad Request</h1>
<p>Your browser sent a request that this server could not understand.<br />
</p>
<hr>
<address>Apache Server at www.debian.org Port 443</address>
</body></html>
closed

You do not even get a chance to enter the “Host: …” line!
The same is true for other servers – as the one supporting e.g. one of my own domains “anracon.de”.

A simple trick shows what the correct request format is and that it works …

Let us use a
simple trick with echo and a pipe on the Linux shell:

myself@mytux:~> (echo -ne "GET / HTTP/1.1\r\nHost: www.debian.org\r\n\r\n") | openssl s_client -tls1_2 -quiet  -connect www.debian.org:443

This leads to

depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
verify return:1
depth=0 CN = www.debian.org
verify return:1
HTTP/1.1 200 OK
Date: Mon, 28 Dec 2020 23:09:01 GMT
Server: Apache
Content-Location: index.en.html
Vary: negotiate,accept-language,Accept-Encoding,cookie
TCN: choice
X-Content-Type-Options: nosniff
X-Frame-Options: sameorigin
Referrer-Policy: no-referrer
X-Xss-Protection: 1
Strict-Transport-Security: max-age=15552000
Upgrade: h2,h2c
Connection: Upgrade
Last-Modified: Sun, 27 Dec 2020 19:27:21 GMT
ETag: "36b1-5b777257b5a41"
Accept-Ranges: bytes
Content-Length: 14001
Cache-Control: max-age=86400
Expires: Tue, 29 Dec 2020 23:09:01 GMT
X-Clacks-Overhead: GNU Terry Pratchett
Content-Type: text/html
Content-Language: en

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html lang="en">
<head>
  <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
  <title>Debian -- The Universal Operating System </title>
...
...
</div>
<!--/UdmComment-->
</div> <!-- end footer -->
</body>
</html>

After a timeout we get back our prompt.

So, we see that the server reacts properly for the end of line characters used, namely “\r\n” after “GET / HTTP/1.1” and after “Host: www.debian.org” plus after an empty line (leading to 2 “\r\n” at the end of the request).

But if we this change to

myself@mytux:~> (echo -ne "GET / HTTP/1.1\nHost: www.debian.org\r\n\r\n") | openssl s_client -tls1_2 -quiet  -connect www.debian.org:443

we run into a “HTTP/1.1 400 Bad Request” answer again
(Note the “\n” before “Host: …”)

Obviously, our “openssl s_client” interface works with LF-characters when pressing the “ENTER”-key on the command line. Well, the interface uses the typical Linux/Unix-style for EOLs …

By the way:
Some servers set very short timeouts for receiving all request lines – you may have difficulties with typing fast enough. Then the “trick” with an echo-command and a pipe is very useful to test the server none the less.

What is the reason for the unequal behavior of different servers?

The correct way to build HTTP-Requests is described e.g. in “www.tutorialspoint.com (http/http_requests.htm)“; I quote:

A Request-line
Zero or more header (General|Request|Entity) fields followed by CRLF
An empty line (i.e., a line with nothing preceding the CRLF) indicating the end of the header fields
Optionally a message-body

According to RFC7230 a HTTP-Request line has the following format:

request-line = method SP request-target SP HTTP-version CRLF

However, in section 3.5 the named RFC also says:

Although the line terminator for the start-line and header fields is the sequence CRLF, a recipient MAY recognize a single LF as a line terminator and ignore any preceding CR.

So, this explains the different behavior of some web-servers to the commands sent with “openssl s_client”.

What can we do with “openssl s_client” to enforce a CRLF as the end of lines when pressing the “ENTER”-key?

The ”
openssl s_client” has a lot of options. You find an overview at the following URI:
https://zoomadmin.com / HowToLinux / LinuxCommand / s_client
The required option for our purpose is “-crlf“; we thus arrive at:

openssl s_client <strong>-quiet -crlf</strong> -tls1_2 -connect YOUR_TARGET_WEB_DOMAIN:443

as the right way to bring RFC compliant web-servers to answer without error messages to manually sent HTTP requests within “openssl s_client”.

Prepare your terminal for long answers from the server

Some web-sites may present long web pages. The HTTP/HTML code sent to you as an answer may be pretty long. As most Linux terminals limit the scroll-able output length by default you should look out for settings that allow for long or infinite output within the terminal emulation of your choice. KDE’s “konsole” e.g. offers you an option to allow for scrolling output with unlimited length. A noteworthy side-effect is, however, that the contents is saved unencrypted in temporary files (for which you can define a location on your PC). So, be a bit careful when using such options.

Conclusion

Using CRLF is the RFC defined way to properly end HTTP request and header field lines. (… whether we from the Linux side may like this MS influenced definition or not …). To enforce an “openssl s_client” to interpret the signal from an “ENTER”-key as “CRLF” (instead of “LF”) we should use the option “-crlf” when opening “s_client”. The additional options “-ign_eof” or “-quiet” are useful to prevent a shutdown of the connection before the server’s answer is fully displayed.

Links

https://zoomadmin.com/HowToLinux/LinuxCommand/s_client
https://stackoverflow.com/questions/5757290/http-header-line-break-style
https://tools.ietf.org/html/rfc7230#section-3.5
https://www.tutorialspoint.com/http/http_requests.htm
https://www.tutorialspoint.com/http/http_quick_guide.htm

Samba 4, shares, wsdd and Windows 10 – how to list Linux Samba servers in the Win 10 Explorer

Posted on 24. May 2020 by eremo

These days I relatively often need to work with Windows 10 at home (home-office, corona virus, …). Normally, I isolate my own Win 10 instance in a VMware virtual machine on my Linux PC – and reduce any network connections of this VM to selected external servers. Under normal conditions all ports on the Linux host are closed for the virtual machine [VM]. But on a few temporary occasions I want to the Win 10 system to access a specific Samba exchange directory on a KVM virtualized Linux instance on the same host.

Off topic: You see that I never present directories of my Linux host directly to a Win 10 guest via Samba. Instead I transfer files via an exchange directory on an intermediate VM whose Samba service is configured to disallow access of the Win system on shares presented to the host. A primitive, but effective form of separation. The only inconvenient consequence is that synchronization becomes a two-fold process on the host and the Linux VM. But we have Linux tools for this, so the effort is limited. )

Of course we want to use the SMB protocol in a modern version, i.e. version 3.x (SMB3), over TCP/IP for this purpose (port 445). In addition we need some mechanism to detect and browse SMB servers on the Windows system. In the old days NetBIOS was used for the latter. On the Linux side we had the nmbd-daemon for it – and we could set up a special Samba server as a WINS server.

During the last year Microsoft has – via updates and new builds of Windows 10 – followed a consistent politics of deactivating the use of SMB V1.0 systematically. This, however, led to problems – not only between Windows PCs, but also between Win 10 instances and Samba 4 servers. This article addresses one of these problems: the missing list of available Samba servers in the Windows Explorer.

There are many contributions on the Internet describing this problem and some even say that you only can solve it by restoring SMB V1 capabilities in Win 10 again. In this article I want to recommend two different solutions:

Ignore the problem of Samba server detection and use your Samba shares on Win 10 with the SMB3 protocol as network drives.
If you absolutely want to see and list your Samba servers in the Windows Explorer of a Win 10 client, use the “Web-Service-Discovery” service via a WSDD-daemon provided by a Python script of Steffen Christgau.

I myself got on the right track of solving the named problem by an article of a guy called “Stilez”. His article is the first one listed under the section “Links” below. I recommend strongly to read it; it is Stilez who deserves all credit in pointing out both the problem and the solution. I just applied his insight to my own situation with virtualized Samba servers based on Opensuse Leap 15.1.

SMB V1.0 should be avoided – but NetBIOS needs it to exchange information about SMB servers

SMB, especially version SMB V1.0, is well known for security problems. Even MS has understood this – especially after the Wannacry disaster. See e.g. the links in the section “Links” => “Warnings of SMBV1” at the end of this article. MS has deactivated SMB V1 in the background via some updates of Win 8 and Win 10.

One of the resulting problem is that we do not see Samba servers in the Windows Explorer of a Win 10 system any longer. In the section “Network” of the Windows Explorer you normally should see a list of servers which are members of a Workgroup and offer shares.

Two years ago we would use NetBIOS’s discovery protocol and a WINS server to get this information. Unfortunately, the NetBIOS service detection ability depends on SMB1 features. The stupid thing is that we for a long while now had and have a relatively secure SMB2/3, but NetBIOS discovery only worked with SMB V1 enabled on the Windows client. Deactivating SMB V1 means deactivating NetBIOS at the
same time – and if you watch your Firewall logs for incoming packets from the Win 10 clients you will notice that exactly such a thing happened on Win 10 clients.

This actually means that you can have a full featured Samba/NetBIOS setup on the Linux side, that you may have opened the right ports on the firewalls for your Samba/WINS server and client systems, but that you will nevertheless not get any list of available Samba servers in Win 10’s Explorer. 🙁

Having understood this leads to the key question for our problem:

By what did MS replace the detection features of NetBIOS in combination with SMB-services?

Settings on the MS Win side – which alone will not help

When you google a bit you may find many hints regarding settings by which you activate network “discovery” functionalities via two Windows services. See

https://www.wintips.org/fix-windows-10-network-computers-not-showing/
https://winaero.com/blog/network-computers-not-visible-windows-10-version-1803/

You can follow these recommendations. If you want to see your own PC and other Windows systems in the Explorer’s list of network resources you must have activated them (see below). However, in my Win 10 client the recommended settings were already activated – with the exception of SMB V1, which I did and do not wish to reactivate again. The “discovery” settings may help you with other older Windows systems, but they do not enable a listing of Samba 4 servers without additional measures on Win 10.

There is another category of hints which in my opinion are contra-productive regarding security. See https://devanswers.co/network-error-problem-windows-cannot-access-hostname-samba/
Why activate an insecure setting? Especially, as such a setting does not really help us with our special problem? 🙁

A last set of hints concerns the settings on the Samba server, itself. I find it especially nice when such recommendations come from Microsoft :-). See: http://woshub.com/cannot-access-smb-network-shares-windows-10-1709/

[global]
server min protocol = SMB2_10
client max protocol = SMB3
client min protocol = SMB2_10
encrypt passwords = true
restrict anonymous = 2

Thanks to MS we now understand that we should not use SMB V1 …. But, actually, these hints are again insufficient regarding the Explorer problem …

What you could do – but should NOT do

Once you have understood that NetBIOS and SMB V1 still have an intimate relation (at least on a Windows systems) you may get the idea that there might exist some option to reactivate SMBV1 again on the Win 10 system. This is indeed possible. See here:
https://community.nethserver.org/t/windows-10-not-showing-servers-shares-in-network-browser/14263/4
https://www.wintips.org/fix-windows-10-network-computers-not-showing/

If you follow the advice of the authors and in addition re-open the standard ports for NetBIOS (UDP) 137, 138, (TCP) 139 on your firewalls between the Win 10 machine and your Samba servers you will – almost at once – get up the list of your accessible Samba servers in the Network section of the Win 10 Explorer. (Maybe you have to restart the smb and nmb services on your Linux machines).

But: You should not do this! SMB V1 should definitely become history!

Fortunately, a re-activation of SMB V1
on a Win 10 system is NOT required to mount Samba shares and it is neither required to get a list of available Samba servers in the Win 10 Explorer.

What you should do: Win 10 service settings

There are two service settings which are required to see other servers (and your own Win10 PC itself) in the list of network hosts presented by the Windows explorer:
Start services.msc ( press the Windows key + R => Enter “services.msc” in the dialog. Or: start services.msc it via the Control Panel => System and Security => Services)

Look for “Function Discovery Provider Host” => Set : Startup Type => Automatic
Look for “Function Discovery Resource Publication” => Set : Startup Type => Automatic (Delayed Start) !!

I noticed that on my VMware Win 10 guests the second setting appeared to be crucial to get the Win 10 PC itself listed among the network servers.

What you should do: Use the SMBV3 protocol!

As you as a Linux user meanwhile have probably replaced all your virtualized Win 7 guests, you should use the following settings in the [global] section of the configuration file “/etc/samba/smb.conf” of your Samba servers:

[global]
…
“protocol = SMB3”.
…

This is what Win 10 supports; you need SMB2_10 with some builds of Win 8 (???), only. Remember also that port 445 must be open on a firewall between the Win 10 client and your Samba server.

For Linux requirements to use SMB3 see
https://wiki.samba.org: SMB3 kernel status
For “SMB Direct” (RDMA) you normally need a kernel version > 4.16. On Opensuse Leap 15.1 most of the required kernel features have been backported. In Win 10 SMB Direct is normally activated; you find it in the “Window-Features” settings (https://www.windowscentral.com/how-manage-optional-features-windows-10)

Not seeing Samba servers in the Explorer does not mean that mounting a Samba share as a network drive does not work

Not seeing the Samba servers in the Win 10 Explorer – because the NetBIOS detection is defunct – does not mean that you cannot work with a Samba share on a Win 10 system. You can just “mount” it on Windows as a “network drive“:

Open a Windows Explorer, choose “This PC” on the left side, then click “Map network drive” in the upper area of the window and follow the instructions:
You choose a free drive letter and provide the Samba server name and its share in the usual MS form as “\\SERVERNAME\SHARE”.
Afterwards, you must activate the option “Connect using different credentials” in the dialog on the Win 10 side, if your Win 10 user for security reasons has a different UID and Password on the Samba server than on Win 10. Needless to say that this is a setting I strongly recommend – and of course we do not allow any direct anonymous or guest access to our Samba server without credentials delivered from a Windows machine (at least not without any central authentication systems).
So, you eventually must provide a valid Samba user name on your Samba server and the password – and there you happily go and use your resources on the Samba share from your Win 10 client.

I assumed of course that you have allowed access from the Win 10 host and the user by respective settings of “hosts allow” and “valid users” for the share in your Samba configuration.
Note: You need not mark the option for reconnecting the share in the Windows dialog for network drives if you only use the Samba exchange shares temporarily.

On an Opensuse system this works perfectly with the protocol settings for SMB3 on the server. So, you can use your shares even without seeing the samba
server in the Explorer: You just have to know what your shares are named and on which Samba servers they are located. No problem for a Linux admin.

In my opinion this approach is the most secure one among all “peer to peer”-approaches which have to work without a central network wide authentication service. It only requires to open port 445 for the time of a Samba session to a specific Samba server. Otherwise you do not provide any information for free to the Win 10 system and its “users”. (Well, an open question is what MS really does with the provided Samba credentials. But that is another story ….)

What you should do: Use the WSDD service on your Samba server

If you allow for some information sharing between your virtualized Win 10 and other KVM based virtual Samba machines in your LAN – and are not afraid of Microsoft or Antivirus companies on the Windows system to collect respective information – then there is a working option to get a stable list of the available Samba servers in the Windows Explorer – without the use of SMB V1.0.

Windows 10 implements web service detection via multiple mechanisms; among them: Multicast messages over ports 3702 (UDP), TCP 5357 and 1900 (UDP). For a detection of Samba services you “only” need ports 3702 (UDP) and 5357 (TCP). The general service detection port 1900 can remain closed in the firewalls between your Win 10 instances and your Linux world for our specific purpose. See
https://www.speedguide.net/port.php?port=5357
https://www.speedguide.net/port.php?port=3702
https://techcommunity.microsoft.com/t5/ask-the-performance-team/ws2008-the-wsd-port-monitor/ba-p/372760
https://en.wikipedia.org/wiki/Simple Service Discovery Protocol

The mechanism using ports 3702 and 5351 is called “Web Service Discovery” and was introduced by MS to cover the detection of printers and other devices in networks. In combination with SMB2 and SMB3 it is the preferred service to detect Samba services.

OK, do we have something like a counter-part available on a Linux system? Obviously, such a service is not (yet?) included in Samba 4 – at least not in the 4.9 version on my system with Opensuse Leap 15.1. The fact that WSD is not (yet?) a part of Samba may have some good reasons. See link.
One can understand the reservations and hesitation to include it, as WSD also serves other purposes than just the detection of SMB services.

Fortunately, a guy named Steffen Christgau, has written an (interesting) Python 3 script, which offers you the basic WSD functionality. See https://github.com/christgau/wsdd.

You can use the script in form of a daemon process on a Linux system – hence we speak of WSDD.

Using YaST I quickly found out that a WSDD RPM package is actually included in my “Opensuse Leap 15.1 Update” repository. People with other Linux distros may download the present WSDD version from GitHub.

On Opensuse it comes with an associated systemd service-file which you find in the directory “/usr/lib/systemd/system”.

[Unit]
Description=Web Services Dynamic Discovery host daemon
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
AmbientCapabilities=CAP_SYS_CHROOT
PermissionsStartOnly=true
Environment= WSDD_ARGS=-p
ExecStartPre=/usr/lib/wsdd/wsdd-init.sh
EnvironmentFile=-/run/sysconfig/wsdd
ExecStart=/usr/sbin/wsdd --shortlog -c /run/wsdd $WSDD_ARGS
ExecStartPost=/usr/bin/rm /run/
sysconfig/wsdd
User=wsdd
Group=wsdd

[Install]
WantedBy=multi-user.target

Reading the documentation you find out that the daemon runs chrooted – which is a reasonable security measure.
Opensuse even provides an elementary configuration file in “/etc/sysconfig/wsdd“.

I used the parameter

WSDD_WORKGROUP=”MYWORKGROUP”

there to announce the right Workgroup for my (virtualized) Samba server.

So, I had everything ready to start WSDD by “rcwsdd start” (or by “systemctl start wsdd.service”) on my Samba server.

On the local firewall of the SMB server I opened

port 445 (TCP) for SMB(3) In/Out for the server and from/to the Win-10-Client,
port 3702 (UDP) for incoming packets to the server and outgoing packets from the server to the Multicast address 239.255.255.250,
port 5357 (TCP) In/Out for the server and from/to the Win 10 client.

And: I closed all NetBIOS ports (UDP 137, 138 / TCP 139) and eventually stopped the “nmbd”-service on the Samba server! (UDP 137, 138 / TCP 139)

Within a second or so, my Samba 4 server appeared in the Windows 10 Explorer!

Further hints:
As the 3702 port is used with the UDP protocol it should be regarded as potentially dangerous. See: https://blogs.akamai.com/sitr/2019/09/new-ddos-vector-observed-in-the-wild-wsd-attacks-hitting-35gbps.html
The port 1900 which appeared in the firewall logs does not seem to be important. I blocked it.

So far, so good. However, when I refreshed the list in the Win 10 Explorer my SAMBA server disappeared again. 🙁

What you should do: Take special care about the network interface to which the WSDD service gets attached to

It took me a while to find out that the origin of the last problem had to do with the fact that my virtualized server and my Win 10 client both had multiple network interfaces on virtualized bridges. There are no loops in the configuration, but it occurred that multiple broadcasts packets arrive via different paths at the Samba server and were answered – and thus multiple return messages appeared at the Win 10 client during a refresh – which Win 10 did not like (see the discussion in the following link.
https://github.com/christgau/wsdd/issues/8

As soon as I restricted the answer of the Samba server to exactly one of the interfaces on my virtual bridge via the the parameter “WSDD_INTERFACES” in the “/etc/sysconfig/wsdd”-configuration file everything went fine. Refreshes now lead to an immediate update including the Samba server.

So, be a little careful, when you have some complicated bridge structures associated with your virtualized VMware or KVM guests. The WSDD service should be limited to exactly one interface of the Samba server.

Note: As we do not need NetBIOS any longer – block ports 137, 138 (UDP) and 139 (TCP) in your firewalls! It will make you feel better instantaneously.

Conclusion

The “end” of SMB V1 on Win 10 is a reasonable step. However, it undermines the visibility of Samba servers in the Windows Explorers. The reason is that NetBIOS requires SMB1.0 features on Windows. NetBIOS is/was therefore consistently deactivated on Win 10, too. The service detection on the network is replaced by the WSD service which was originally introduced for printer detection (and possibly other devices). Activating it on the Win 10 system may help with the detection of other Windows (8 and 10) systems on the network, but not with Samba 4 servers. Samba servers presently only serve NetBIOS requests of Win clients
to allow for server and share detection. Therefore, without additional measures, they are not displayed in the Windows Explorer of a regular Win 10 client.

This does, however, not restrict the usage of Samba shares on the Win 10 client via the SMB3 protocol. They can be used as “network drives” – just as before. Not distributing name and device information on a network has its advantages regarding security.

If you absolutely must see your Samba servers in the Win 10 Explorer install and configure the WSDD package of Steffen Christgau. You can use it as a systemd service. You should restrict the interfaces WSDD gets attached to – especially if your Samba servers are attached to virtual network bridges (Linux bridges or VMware bridges).

So:

Disable SMBV1 in Windows 10 if an update has not yet done it for you!
Set the protocol in the Samba servers to SMBV3!
Try to work with “networks drives” on your Win 10 guests, only!
Install, configure and use WSDD, if you really need to see your Samba servers in the Windows Explorer.
Open the port 445 (TCP, IN/OUT between the Win 10 client and the server), 3072 (UDP, OUT from the server and the Win 10 client to 239.255.255.250, IN to the server from the Win 10 client / IN to the Win 10 client from the server; rules details depending on the firewall location), port 5357 (TCP; In/OUT between the Samba server and the Win 10 client) on your firewalls between the Samba server and the Win 10 system.
Close the NetBIOS ports in your firewalls!
You should also take care of stopping multicast messages leaving perimeter firewalls; normally packets to multicast addresses should not be routed, but blocking them explicitly for certain interfaces is no harm, either.

Of course you must repeat the WSDD and firewall setup for all your Samba servers. But as a Linux admin you have your tools for distributing common configuration files or copying virtualization setups.