More fun with veth and Linux network namespaces – IV – L2-segments, same IP-subnet, ARP and routes

In the course of my present series on veths, network namespaces and virtual VLANs

More fun with veth and Linux network namespace – III – L2-segments of the same IP-subnet and routes in coupling network namespaces

More fun with veth and Linux network namespace – II – two L2-segments attached to a common network namespace

More fun with veth and Linux network namespace – I – open questions

we have started to study a somewhat academic network configuration:

We have attached two L2-segments to a common network namespace, with all IPs of both segments belonging to one and the same IP-subnet class.

The IP-setup makes the scenario a bit peculiar. It is not a situation an admin would normally create. Instead the IPs of each segment would be members of one of two distinct and different IP-subnets.

However, our scenario already taught us some important lessons to keep in mind when we approach our eventual objective:

Sooner or later we want to answer the question how to configure virtual VLAN configurations, in which veth-subdevices for tagged VLAN lines terminate in a common and routing namespace.

By our academic scenario we found out that we need to set up much more specific routes than the standard ones which are automatically created in the wake of “ip addr add” commands. Despite the fact that forwarding was disabled in the coupling network namespace! Only with clear and fitting routes we got a reasonable behavior of our artificial network with respect to ICMP- and ARP-requests in the last post (III).

Well, this is, in a way, a platitude. Whenever you have a special network you must adapt the routes to your network layout. This, of course, includes the resolution of ambiguities which our scenario introduced. The automatically defined routes were insufficient and caused an asymmetry in the ICMP-replies in the otherwise very symmetric configuration (see the drawing below).

Nevertheless, the last post and also older posts on veths and virtual VLANs triggered an intense discussion of two of my readers with me concerning ARP. In this post I, therefore, want to look at the impact of routes on ARP-requests and ARP-replies between the namespaces in more detail.

The key question is whether and to what extend the creation and emission of ARP-packets via a particular network interface may become route-dependent in namespaces (or on hosts) with multiple network interfaces.

While we saw a clear and route dependent asymmetry in the reaction of the network to ICMP-requests, we did not fully analyze how this asymmetry showed up on the ARP-level.

In my opinion we have already seen that ARP-requests triggered within ICMP-requests showed some asymmetry regarding the NIC used and the segment addressed – depending on the defined routes. But the experimental data of the last post did not show the flow of ARP-replies in detail. And we only regarded ARP-requests triggered by ICMP-requests. I.e. we watched ARP-requests created by the Linux-kernel to support the execution of ICMP-requests, but not pure and standalone ARP-requests.

Therefore, it is still unclear

  • whether routes have an impact on pure standalone ARP-requests, i.e. ARP-requests not caused by ICMP-requests (or by other requests of higher layer protocols),
  • whether routes do have an impact on ARP-replies.

The experiments below will deliver detailed information to clarify these points.

When I speak of ARP-tables below, I am referring to the ARP-caches of the various network namespaces in our scenario.

Continue reading

More fun with veth and Linux network namespaces – II – two L2-segments attached to a common network namespace

In my first post of my new series about virtual networking

More fun with veth and Linux network namespace – I – open questions

I collected some questions which had remained open in an older post series of 2016 about veths, unnamed network namespaces and virtual VLANs. In the present series I will try to answer at least some of these questions.

In the present post we will study a scenario with the following basic properties:

Two L2-segments (based on two Linux bridges), each with two attached NICs in separate network namespaces, will be connected by veths to a further common network namespace. The IPs of all NICs and relevant veth end-points will be members of one and the same IP subnet class (a class C net).

No VLANs or firewalls will be set up. So, this is a very plain and seemingly simple scenario: Two otherwise separate L2-segments terminate with border NICs in a common network namespace.

But note that our scenario is different from the typical situation of a router or routing namespace. The reason is that our L2-segments and their respective NICs do not belong to different logical IP networks with different IP broadcast regions. We have just one common C-class IP-net and not two different ones.

In this second post we will first try to find out by theoretical reasoning what is required to enable communication between the common namespace and each of the segments. Meaning separate communication; no forwarding between the segments. We shall also study ARP requests and replies under these conditions. In a forthcoming third post we will verify our ideas by concrete experiments.

In future posts we will afterward try to describe what may be required in addition to enabling forwarding in the common network namespace to establish a cross-segment communication. I.e., we want to find out what we must do to ensure that a namespace (host) in one segment can talk to namespaces (hosts) in the other segment.

I will use the abbreviation “netns” for network namespaces throughout this post.

Scenario SC-1: Two L2-segments coupled by a common and routing namespace

Let us first look at a graphical drawing showing our scenario:

A simple way to build a virtual L2-segment is the following:

We set up a Linux bridge (e.g. brB1) in a dedicated network namespace (e.g. netnsB1). Via veth devices we attach two further network namespaces (netns11 and netns12) to the bridge. You may associate the latter namespace with virtual hosts reduced to elementary networking abilities. As I have shown in my post series of the year 2016 we can enter such a namespace and execute commands there. The veth endpoints in netns11 and netns12 get IP addresses.

We build two of such segments, S1 and S2, with each of the bridges located in its own namespace (netnsB1 and netnsB2). Then we use veth devices again to connect the two bridges (= segments) to a common namespace (netnsR).

The graphics shows that we all in all have 7 network namespaces:

  • netns11, netns12, netns21, netns22 represent hosts with NICs and IPs that want to communicate with other hosts.
  • netnsB1 and netnsB2 host the bridges.
  • netnsR is a namespace where both segments, S1 and S2, terminate – each via a border NIC (veth-endpoint).

The sketch makes it clear that netns11 will certainly be able to communicate with netns12. The same holds for netns21 and netns22.

netnsR is the namespace which is most interesting in our scenario: Without special measures packets from netns11 will not reach netns21 or netns22. So, we have indeed realized two separated L2-segments S1 and S2 attached to a common network namespace.

But we cannot be so sure what will happen with ARP and ICMP request and/or answering packets send from netnsR to one of the four namespaces netns11, netns12, netns21, netns22. I come back to this point in a minute.

Regarding IP addresses: Outside the bridges we must assign IP addresses to the respective veth-endpoints. As said: During the setup of the devices I use IP-addresses of one and the same class C network. All IPs belong to the same IP-subnet: 192.168.5.0/24. The bridges themselves do not need assigned IPs. In our scenario they could have been replaced by an Ethernet bus cable with outtakes.

Side remark: Do not forget that a Linux bridge can in principle get an IP address itself and work as a special NIC connected to the bridge ports. We do, however, not need or use this capability in our scenario.

Theoretical analysis of the situation of and in netnsR

Both L2-segments terminate in netnsR. The role of netnsR is basically similar to that of a router, but with more ambiguity and uncertainty because both border NICs belong to the same IP-net.

Continue reading

More fun with veth and Linux network namespaces – I – open questions

In 2016 I wrote a series of posts about a special section of Linux-based virtual networking, namely network namespaces and virtual VLANs. In parallel I published another series about IPtables rules on virtual Linux bridges. See
Fun with veth-devices, Linux bridges and VLANs in unnamed Linux network namespaces – I
and
Linux bridges – can iptables be used against MiM attacks based on ARP spoofing ? – I
plus related posts.

The motivation behind those post series was to find ways to define and separate packet paths in complex virtual LANs, which we may create on virtualization hosts. VLAN tagging is one of the methods to define certain allowed and disallowed paths of ICMP and TCP/IP packets between virtual hosts, virtual LAN segments or virtual network namespaces on Linux hosts. On virtual bridges/switches additional iptables/ebtables or nftables rules may help to control the traffic between certain bridge-ports and the segments behind them.

This new post series extends my old series. Among other things I will have a closer look on scenarios in which two separate LAN- and VLAN-segments with NICs in one and the same IP-(sub)-net (class C network) get connected by a namespace with routing rules. In addition the behavior of ARP and ICMP packets on bridges with IPtables rules will be analyzed.

I want to mention that some really clever and serious questions of a reader, Joshua Greenberg, motivated me to do this work. His questions regarding some of my old statements gave me some headache – in particular with respect to ARP. We agreed upon that some question marks behind my line of thought of 2016 could only be answered by further experiments. So, many thanks to Jushua for giving me a push to turn my head towards virtual networking again.

Open points and questions regarding the old post series

Both named post series on virtual networking got a long way, but were not finalized. The connection of (virtual) VLANs to external real (physical) networks via namespaces that contain one or more real NICs were not discussed. It is not at all clear whether we can separate the traffic between the VLANs and the outer world without the help of firewalls. In this context we may also have to look closer at the relation of VLANs to IP-subnets.

I briefly outline some of further open questions.

Multiple VLAN termination points and related sub-devices of one veth-endpoint in a network namespace

I admit that I should have analyzed virtual veth connections which support multiple VLANs more thoroughly for some of my scenarios in the 2016 posts. In particular a closer look at scenarios where a veth-endpoint puts multiple VLAN-related sub-devices into one and the same target namespace would have been helpful to avoid confusion. I.e., I should study scenarios with different virtual VLANs terminating in a common namespace more carefully. Before you think this gets boring, note that the veth end-point device itself gets just one IP. See e.g. post.

Such a situation introduces multiple open tagging virtual NIC devices in one namespace. Somehow we need to control which VLAN device and path a packet should “choose” to get the right VLAN tag (VID) and to enable its further transport through the right VLAN to a target NIC. And we should better understand what ARP packets do in the Linux network namespace in such a situation.

Such a scenario is e.g. also of interest when traffic of different virtual VLANs must be directed to or through a common namespace which comprises VLAN end-points and a real NIC; the latter to enable communication to the outside world of a Linux virtualization host. Such a namespace would be a routing one.

In this context the coupling of network layer 2 (Link layer) to layer 3 (IP or network layer) is of special interest. Layer coupling via deliverance of information about IP/MAC-relations is done by the ARP protocol. We all know that ARP is often used in hacker attacks. So it might be a good idea to know what happens with ARP in namespaces which contain multiple VLAN end-points.

In my old posts I speculated that ICMP and ARP answers in such unclear situations may depend on defined routes in the namespace. We shall find out in how far this is true by two corresponding experiments.

Connecting different L2-segments without and with VLANs by routing network namespaces

The scenario named above is just a special case of connecting or separating “L2-segments” in a common namespace. I define a L2-segment as a LAN-segment, in which packets on the Link Layer travel freely. A L2-segment forms an Ethernet broadcast domain; Ethernet broadcast packets reach all NICs attached to a L2-segment. A good introduction to L1- and L2-segments and related Ethernet broadcasts is given here. Note that a complexly and hierarchically structured L2-segment may be created by connecting real or virtual linear Ethernet bus-lines by Linux bridges.

An exciting area of unusual scenarios opens up when so called L2-segments do not coincide with logical IP sub-nets.

Two separated L2-segments may have NICs with IPs belonging to one and the same IP network class. The attentive reader of my 2016 series has of course noted that this was the case in all scenarios discussed at that time. Actually, this was a clue: I used VLANs to isolate packet paths within one and the same IP sub-net ( a class C net) and within originally coherent L2-segments against each other.

Now, multiple different and originally separated virtual L2-segments with IPs of the same IP net may be coupled by routing devices, routing namespaces or (VLAN-aware) bridges/switches. What happens with ARP requests and answers in such situations?

On the other hand side NICs attached to one and the same L2-segment may belong to different IP sub-nets. Which on first sight appears to be a stupid mis-configuration; but it may occur. What about ARP then?

What about VLANs across segments belonging to different logical IP sub-nets? Do such configurations make sense at all?

Bridges, iptables, VLANs and ARP

Readers have also sent me questions regarding VLAN-aware bridges and the propagation of ARP requests and ARP answer packets when IPtables rules control the packet traffic between bridge ports via “physdev“-related commands. What about tagged packets on a bridge with IPtables?

Summary

Linux network namespaces, virtual Linux bridges and veth network devices make it possible to realize complex virtual network scenarios on a virtualization host – including virtual VLANs. As such virtual networks must be well protected against hacker attacks as real networks we should first understand packet transport through virtual networks, their devices and in particular across Linux bridges sufficiently well.

Linux network namespaces and veths allow us to study packet transfer on different OSI layers in elementary network scenarios for L2-segments with and without virtualized VLANs in detail.

The questions which remained open in my old post series and some new questions of readers invite us to study a bunch of further scenarios in a new post series.

In the next post

More fun with veth and Linux network namespace – II – two L2-segments attached to a common network namespace

I will pose and study a scenario without VLANs and respective tags. I will just attach veth end-points of two otherwise separated L2-segments to a common network namespace. Nevertheless, this very simple experiment will shed some light on open questions regarding routes, ARP and ICMP requests and answers. It will also lead us to aspects of PROXY ARP in (routing and forwarding) network namespaces.