More fun with veth, network namespaces, VLANs – IV – L2-segments, same IP-subnet, ARP and routes

Posted on by

In the course of my present post series on veths, network namespaces and virtual VLANs

we have studied a somewhat academic network scenario:

Two L2-segments were attached to a common network namespace, with all IPs of both segments belonging to one and the same IP subnet class. The network traffic was analyzed for different route configurations. Only detailed routes ensured a symmetric handling of packets in both segments.

The IP-setup makes the scenario a bit peculiar. It is not a situation an admin would normally create. Instead the IPs of each segment would be members of one of two distinct and different IP-subnets.

However, our scenario already taught us some important lessons to keep in mind when we approach our eventual objective:

Sooner or later we want to answer the question how to configure virtual VLAN configurations, in which veth subdevices for tagged VLAN lines terminate in a common and routing namespace.

By our academic scenario we found out that we need to set up much more specific routes than the standard ones which are automatically created in the wake of “ip addr add” commands. Despite the fact that forwarding was disabled in the coupling network namespace! Only with clear and fitting routes we got a reasonable behavior of our artificial network with respect to ICMP- and ARP-requests in the last post (III).

Well, this is, in a way, a platitude. Whenever you have a special network you must adapt the routes to your network layout. This, of course, includes the resolution of ambiguities which our scenario introduced. The automatically defined routes were insufficient and caused an asymmetry in the ICMP-replies in the otherwise very symmetric configuration (see the drawing below).

Nevertheless, the last post and also older posts on veths and virtual VLANs triggered an intense discussion of two of my readers with me concerning ARP. In this post I, therefore, want to look at the impact of routes on ARP-requests and ARP-replies between the namespaces in more detail.

The key question is whether and to what extend the creation and emission of ARP-packets via a particular network interface may become route-dependent in namespaces (or on hosts) with multiple network interfaces.

While we saw a clear and route dependent asymmetry in the reaction of the network to ICMP-requests, we did not fully analyze how this asymmetry showed up on the ARP-level.

In my opinion we have already seen that ARP-requests triggered within ICMP-requests showed some asymmetry regarding the NIC used and the segment addressed – depending on the defined routes. But the experimental data of the last post did not show the flow of ARP-replies in detail. And we only regarded ARP-requests triggered by ICMP-requests. I.e. we watched ARP-requests created by the Linux-kernel to support the execution of ICMP-requests, but not pure and standalone ARP-requests.

Therefore, it is still unclear

  • whether routes have an impact on pure standalone ARP-requests, i.e. ARP-requests not caused by ICMP-requests (or by other requests of higher layer protocols),
  • whether routes do have an impact on ARP-replies.

The experiments below will deliver detailed information to clarify these points.

When I speak of ARP-tables below, I am referring to the ARP-caches of the various network namespaces in our scenario.

The scenario

The scenario is visualized by the following drawing:

We see that there are two L2-segments, S1 and S2. These segments are attached to one common namespace “netnsR” by border-NICs, namely “veth1R” and “veth2R”. By the symmetry of the whole arrangement we have created a structural ambiguity in netnsR regarding the choice of a segment to send ICM- requests to and/or in which of the two segments (S1 and S2) Ethernet broadcasts for ARP-requests should be created.

In a previous post I have provided commands to set up this scenario on a Linux host. We have a bridge in each segment. The network connections are realized by virtual veth devices and their endpoints. The basic setup commands were fully symmetric with respect to the elements of the segments.

Via using the command “ip addr add .. ” IP-addresses were assigned to all veth endpoints (NICs). As usual, the IP-assignments also led to the automatic creation of simple routes in all namespaces.

In netnsR two routes were created on the same level of IP-subnet discrimination (in both cases the same network 192.168.5.0/24) – but for different interfaces (veth1R and veth2R):

The first of these routes was preferred by the Linux-kernel during previous experiments. Due to selection routes of the Linux kernel. To make things clearer in the forthcoming experiments we will work with with the first route in netnsR, only. I will call this route via veth1R “Route I“. Note that this route-setup creates an asymmetry in netnsR.

We will also focus on the namespaces netns12, netnsR and netns22. It is clear that both ICMP- and ARP-requests sent from either netns12 or netns22 will reach netnsR. More problematic is the path of packets going out from netnsR. We will first use ICMP-requests to trigger ARP-requests and follow the reaction of netnsR to these requests. But in a special experiment we will later also track “stand-alone” ARP-requests which we create via the “arping“-command on the command line.

To see what happens in our (unnamed) namespaces we will start tcpdump in special terminals, from which we access the namespaces by the help of the command “nsenter“. ICMPv6 and IPv6 are deactivated in all of the shown namespaces.

No default router, deactivated forwarding and deactivated Proxy ARP

No default router is defined in netns12 or netns22. Forwarding in netnsR is deactivated. Also Proxy ARP is deactivated in netnsR.

Empty ARP-caches as a starting point

After having set up the scenario, all ARP tables of the involved network namespaces are initially empty:

ARP and routes – theoretical and plausible considerations

One of the central points that came up in the discussions with my readers was that from a formal point of view ARP is assigned to layer 2 – the Link layer (RfC 1122). So, one could argue that any kind of route dependence during the execution of ARP would set a question mark behind this formal assignment.

After some back and forth in the discussions I think it is more a question of perspective. If one accepts

  • that getting information about a MAC for an IP has a purpose – either to resolve tasks on the IP-level (or higher layers) or to support port-related tasks on bridges/switches
  • and that ARP-requests can only reach NICs within the borders of a network segment,

things fall into place again. In my opinion the following basic points provide a guideline to reach solid ground again for the interpretation of what happens in our scenario:

  • ARP supports protocols on higher layers of the TCP/IP stack.
  • ARP supports the maintenance of “forward databases” on layer-2 switches/bridges.
  • ARP works within L2-segments. Reason: Its requests are based on Ethernet broadcasts.
  • ARP’s logic for standard L2-segments (not VLAN-segments) is simple: An ARP-request emitted into an Ethernet segment is answered by a receiving NIC in that segment if this NIC has the IP for which a MAC was requested. The reply normally is sent as an Ethernet unicast. If no NIC in the segment has the IP the requesting NIC gets no reply.
  • An entry for an IP/MAC/NIC is inserted or updated in an ARP table (= ARP cache) of a namespace when an ARP-request arrives there. An entry in the ARP-cache of the requesting namespace is inserted/updated with the information carried by an ARP-reply if the reply is regarded as solicited. Unsolicited ARP-replies are ignored by modern Linux-kernels, if they are not explicitly told to accept them.
  • ARP is not a protocol that by itself finds IPs in a complex LAN with multiple segments, routers and gateways. ARP-requests are not forwarded. Proxy ARP, if activated on routers, may however answer with a MAC of the routing namespace (or host) if the requested IP is known to reside outside the attached requesting segment.

From the perspectives both

  • of routers with multiple network interfaces and multiple attached L2-segments
  • and of hosts (or network namespaces) located in a segment,

the question where an IP is located is in general decided by routes. For IP destinations outside its segment a host would address a router – if the host ever got the router’s IP.

Plausible consequences for namespaces with multiple NICs are:

  • If a standard router (or a forwarding namespace with multiple NICs) needs to manage and direct a protocol request on layer 3 or 4 of the TCP/IP-stack and, therefore, wants to know which of the multiple available interfaces it should use to send Ethernet packets with the protocol’s payload, it has to trust in its routes – if it gets no information otherwise.
  • If a non-forwarding namespace with multiple NICs needs to find out which MAC the NIC with the destination IP has, it must trigger an ARP-request (Ethernet broadcast) in the relevant segment among all of its attached segments. Its only chance to determine the relevant border-NIC is the information given by routes – as long as there is no other instruction about which NIC to use.

You could argue that a namespace like netnsR could check and investigate all of the attached segments. But this would not be economical and could become a nightmare if conflicting information arrived. “Guessing” is no reasonable option either. Therefore, in case of an ambiguity and when a user of who enters network commands has not specified interfaces, it is at least plausible to assume the following:

If ICMP on a host or on a router wants to know which Ethernet MAC to address (if the ARP-table does not have the right information yet), it will trigger an ARP-request in the segment it “thinks” to be the right one according to its set routes.
If a user sends a pure ARP-request (e.g. with the arping-command) the namespace will also use the information given by a route.
The relevant route among multiple defined ones will be selected by standard selection rules of the Linux-kernel.

On routers with multiple NICs a route information should therefore be unique and sufficient for a given IP. In cases where L2-segments coincide with distinct and different IP-subnets, this would not surprise us.

That is it. (At least for standard L2-segments and no VLAN-segments). In my opinion the arguments and “rules” given above explain all of the observations made during the experiments of my last post and also of the experiments presented below.

Actually, I will use the rules discussed above to formulate expectations for the outcome of the experiments posed and analyzed below. We will indeed see that route information is used when the sending network namespace otherwise had to guess which NIC and segment to address. This is even true for pure ARP-requests.

Addendum and reservation, 14.03.2024: All considerations relate to standard L2-segments. Most of our rules, but not all, will be valid for VLAN-segments, too. But in particular, the idea that routes do not have an impact on the creation of ARP-replies will have to be tested for namespaces with attached VLAN-segments based on veth-subdevices and the 802.1q standard for tagged packets. And the idea might very turn out to be questionable or even wrong for some VLAN-/veth-configurations in future posts.

Get arping for some experiments

To be able to create ARP-requests from userspace we need the arping-command and respective libraries. Most Linux distribution offer a package which contains the arping tool. On present Opensuse Leap 15.5 and SLES 15.5 it is available via the “iputils” package.

Opensuse Leap and SLES: If you experience problems with the a tcpdump or tshark analysis of ARP packets created by arping, verify that you got the latest versions of iputils and libpcap1 (or libpcap) from the same repository.

Experiment 1: Ping from netns12 to netnsR and resulting ARP

We send a ICMP-request from netns12 to IP 192.168.5.200 in netnsR.

Expectations:
From the rules above and due to the empty ARP-cache in netns12 we expect that the ICMP-request triggers an ARP-request first. This ARP-request reaches netnsR and causes an ARP-entry in the ARP-cache there. The ARP request should be answered by netnsR because veth2R has the requested IP. Whether the ARP-reply leads to an entry in the ARP table of netns12 has to be checked. It depends on whether the reply is considered to be unsolicited.

As netns12 now has the destination MAC the ICMP-request should be sent by netns12. We expect an ICMP-reply by 192.168.5.200 afterward. Reasons: Route I specifies a correct interface of netnsR to use, namely veth1R. The (probably) already created entry in the ARP-cache in netnsR also provides the right destination MAC in the segment behind veth1R.

Results:

And in netnsR we get (sorry, that I by accident cut off the netnsR information in the screenshot):

We see that an additional, but unsurprising ARP-request is triggered by veth1R (192.168.5.100) after the ICMP-transaction. No request reaches netns22:

The ARP-caches in netns12 and netnsR got the following entries:

Obviously, the ARP-reply to netns12 caused an entry for 192.168.2.200 in the ARP-cache of netns12. The entry for 192.168.5.100 resulted from the late ARP-request from netnsR.

netnsR also learned about the MAC of 192.168.5.11:

We have learned by this experiment that ICMP-requests indeed trigger ARP-requests to resolve unknown MACs, which required to send the ICMP packets. As routes and ARP information in netnsR point to the right segment, netns12 gets its ICMP-reply.

But we can not yet be sure whether there is an impact of the route on the ARP-reply sent to netns12. But experiments 3 and 10 below will show that this not the case.

Experiment 2: Ping from netns12 to netns22 and resulting ARP

Expectations:
The initial ARP-request will not get any reply as there is no NIC in segment S1 with the right IP. The ARP-request is not forwarded to S2. netnsR is not specifically addressed by a unicast packet as the responsible router either, because there is no respective router-entry in netns12.

Results:

The entries in the ARP-cache of netnsR did not change during the experiment (see the last command below):

This is fully consistent with our expectations.

An interesting questions is: Would the requests have worked, if we had defined 192.168.5.100 as the default router address in netns12? a positive answer would at least require that netnsR could resolve the MAC for 192.168.5.22 and send both ARP- and IP-(request-) packets through veth2R. But see experiment 6. The answer actually is NO with the currently set route I.

Experiment 3: Testing asymmetry by ICMP-request from netns22 to netnsR

In the last post we saw a clear asymmetry in the reaction of netnsR to ICMP-requests from S1 and S2. We test this once again and track the generated ARP-packets.

In particular, we want to find out whether an initial ARP-request from netns22 gets an answer or not. If it gets one, we may safely conclude that the defined route in netnsR was ignored for the direction of the ARP-reply and that only the information in the ARP-request packet and the data of the NIC, at which the request arrived, were used to send the ARP-reply.

Expectation: According to our rules discussed above we expect that an initial ARP-request will be sent from netns22 – and that this request will indeed be answered! We should also get a valid entry for 192.168.5.22 in the ARP table of netnsR – for interface veth2R. The subsequent ICMP-request from netns22 should reach netnsR. To answer it netnsR would, however, follow its route definitions, i.e. Route I. Reason: We work on layer 3 now! Therefore the ICMP-request of 192.168.5.22 will NOT be answered:

  1. Despite the fact that netnsR has a valid MAC for 192.168.5.22, already, netnsR would want to send the ICMP-answer to S1 according to the defined Route I. It will therefore send an ARP-request for 192.168.5.22 (!!) into S1 via veth1R.
  2. netnsR will get no answer for an ARP-request for 192.168.5.22 behind veth1R.

Note that the key assumption here is that “Route I” dictates in netnsR where to look for a MAC of 192.168.5.22 to afterward create and send the ICMP-answer. The route points to veth1R and S1 – for this interface there is no valid entry in the ARP-table. Thus we get an ARP-request for 192.168.5.22 behind veth1R. The valid entry for 192.168.5.22 behind veth2R is ignored.

Results:

The ARP-request was indeed answered. This is just how ARP works: netnsR checked whether it had the requested IP (yes, it had) and answered directly to the MAC from where the request came – via the interface the request came from. In consistency with the ARP-cache entry that resulted from the incoming request (see below). Any route information was obviously ignored during the creation of ARP-replies and the transmission of the respective Ethernet unicast packet.

We should also see this in netnsR:

No impact of routes here regarding the ARP-reply to the request from netns22. This should give at least one of my discussion partners some comfort again. All the ARP-communication remains in a sense within segment S2. No ambiguity here. And aside of checking an IP no reference to layer 3. But …

But there is another key result: After having replied to the ARP-request, the ICMP-request from netns22 arrives at netnsR. And now netnsR operates on layer 3 – and it takes its route information seriously. Namely: It wants to send an ICMP-reply packet on this layer to an IP in 192.168.5.0/24 via veth1R.

But there is no valid entry for 192.168.5.22 and veth1R (!) in the ARP table of netnsR. The conflicting information regarding veth2R is ignored. Which is, in my opinion, correct as the configuration could have changed since the last update of the ARP table.

So, netnsR tries to find a valid MAC for 192.168.5.22 behind veth1R. But the corresponding ARP-requests, of course, remain unanswered. This is also reflected by the data of netns12:

So, no ICMP-reply can be sent.

Note that the choice of a NIC and a related segment according to a route was in this experiment done to support ICMP.

In this particular sense the choice of a segment and a respective interface (among multiple available ones) in netnsR to generate and send ARP-requests (in support of higher protocol tasks) is really determined by routes.

However, it is the destination IP which counts in the end. We, therefore, have at least some reason to believe that als a pure ARP-request may be influenced by a route. But see experiment 9.

What about the ARP-caches in the present experiment?

Consequently, we find two entries in netnsR’s ARP table for IP 192.168.5.22: A valid one regarding interface veth1R. This was created in course of the first ARP request coming in from netns22. The second incomplete one is due to the unanswered request for 192.168.5.22 emitted into segment S1.

The asymmetric impact of our (asymmetric) route in netnsR on ICMP is very clear.

Intermediate summary

So far we have seen that ARP-replies were not influenced by routes. But the emission of ICMP-answers and secondarily triggered ARP-requests was obviously controlled by the defined route in netnsR (with its structural ambiguity regarding the choice of interfaces).

The defined “Route I” caused an asymmetry with respect at least to ICMP. We have also seen that an ARP-reply received in the course of an ICMP-transaction causes an entry in the ARP-cache of the namespace which sent the ARP-request.

So, we still have to investigate the following cases:

  • ICMP-requests and secondarily triggered and related ARP-requests from netnsR to netns12 and netns22.
  • Pure ARP-requests sent by a user from netnsR to IPs in netns12 and netns22. Does the route have an impact on such requests?
  • The effect of ARP-replies to pure ARP-requests concerning an update of the ARP-cache in the requesting namespace.

Deletion of all entries in all ARP tables

Before proceeding we delete all entries in all ARP cache tables. This can be done with the help of the command “arp -d“.

Interestingly, for some entries in netnsR we may have to use the “-i” option in addition and provide the interface. (See the man pages for more information.) The entries in question actually are those ones in conflict with the route. So, the route even influences how we have to delete ARP cache entries. Try it yourself if you do not believe it. 🙂

Experiment 4: Ping from netnsR to netns12

With empty ARP tables we now send an ICMP ping from netnsR to 192.168.5.11 in netns12.

Expectations:
According to our previous results and due the empty ARP table and the defined route in netnsR we expect that an initial ARP-request is sent into S1 via veth1R (192.168.5.100). This ARP-request will lead to an entry in the ARP table of 192.168.5.11. An ARP-reply will be sent back to 192.168.5.100. Afterward a packet with the ICMP-request is sent and directly answered by netns12. Reason: Meanwhile netns12 has got the required information to which MAC it must send its ICMP-reply to.

We see that the ARP information from the first arriving ARP-request in netns12 is verified some time after the ICMP-reply.

The ARP entries follow the ARP-requests and ARP-replies.

Experiment 5: Test asymmetry by a ping from netnsR to netns22

We now try to send an ICMP-request from netnsR to netns22. If “Route I” in netnsR has an impact we should see it now clearly.

Expectations:
According to experiment III and our theoretical considerations we expect the route to have an impact on the initial ARP-request which netnsR must send because it has no MAC information for 192.168.5.22. The ARP-request (required to start the ICMP-transaction) would be sent to S1 via veth1R. And get no answer. Thus an Ethernet packet containing the ICMP-request cannot even be sent by netnsR.

The initial ARP-request is indeed sent into S1 via veth1R. And, of course, remains unanswered.

We see that no ARP request reaches netns22.

As expected netns12 cannot answer the ARP request:

Consequently, the ARP entry for 192.168.5.22 behind veth1R remains “incomplete”.

We have learned know that ARP-requests triggered by ICMP-requests are indeed send into a specific segment via an interface (among two available ones) which is defined by a route.

Experiment 6: Ping from netns22 to netnsR

This experiment just confirms experiment 3. However, we send an ICMP-request to 192.168.5.200 now (and not to 192.168.5.100 as before).

Expectations:
An initial ARP-request for 192.168.5.200 will be sent from netns22. Reason: The ARP table of netns22 is empty, still. This request will get a reply. So, the ARP table in netnsR will afterward have a valid entry for 192.168.5.22 and veth2R. Nevertheless, the subsequent ICMP-request from netns22 will remain unanswered because the route forces netnsR again to find a MAC for 192.168.5.22 behind veth1R – see experiment 5.

Note that the ICMP-request just remains unanswered.

netnsR declares 192.168.5.22 to be unreachable – according to its route and missing ARP- replies. Despite a valid entry for 192.168.5.22 in its ARP-cache!

We have again learned that the insufficient and one-sided route in netnsR leads to an ignorance of conflicting entries for the destination IP in the ARP-cache.

Experiment 7: Check security via ping from netns12 to netns22

This experiment and the next one should just show that ARP requests are not exchanged between segments. This is required as a basic security measure.

Expectation:
Under the present conditions ARP-requests from S1 should get no reply and should not appear in netns22. Note: An activated Proxy ARP in netnsR would lead to replies, but we will cover this in another post.

Experiment 8: Check security via ping from netns22 to netns12

Expectation:
Under the present conditions ARP requests should get no reply and should not appear in netns12. Note: An activated Proxy ARP in netnsR may lead to answers, but we will cover this in another post.

Experiment 9: Sending a direct ARP request from netnsR

We now come to an important experiment.
After all of the experiments above you may still argue: Well, all my ARP-requests so far were triggered by an ICMP-request. What if we sent pure ARP-requests directly from netnsR? Will the route “Route I” then still have an impact on the NIC used for the emission of the respective Ethernet broadcast?

The answer may also depend on whether one specifies the network interface to use for sending the request – or not.

To get to results, we set up the whole scenario configuration from scratch again. Afterward we use the “arping“-command to create and send ARP-requests from the command line.

Expectation:
My hypothesis is that the route defined in netnsR indeed has an impact regarding the selection of a network interface and thus of the segment in which the Ethernet broadcast for the ARP-request will be created.

Experiment 9.1: ARP-request from netnsR for 192.168.5.11 and 192.168.5.22 without interface specification

Expectations:
We expect that the route defines the segment to which the ARP-requests are sent. In case of 192.168.5.11 we expect an ARP-reply, in the case of 192.168.5.22 no reply will come back to netnsR. We are, however, unsure whether the ARP-reply will trigger an entry in netnsR’s ARP-cache.

The first ARP-request is answered, for the second no reply is registered. The reason becomes clear from the next screenshots.

We also make a strange observation: The ARP-cache of netnsR remains empty despite the received ARP-reply.

In the screenshots below the records resulting from the two different requests are separated by an empty line.

Obviously, interface veth1R was chosen in both requests and both requests were sent to segment S1! This is consistent with our suspicion that the route determines the NIC used for creating the Ethernet broadcast for the ARP-request.

When we look at the ARP-cache of netns12 we find:

This shows us that arriving ARP-requests lead to an update of the ARP-cache. But seemingly, arriving replies to user initiated ARP-requests in netnsR do not cause an entry in netnsR’s ARP-cache. If this is true an ARP-request from netns12 to netnsR should trigger an ARP-cache update in netnsR. And indeed:

And:

We will give a reason for this in a separate main section below.

Experiment 9.2: ARP-request for 192.168.5.22 and valid entry in the ARP-cache

In this experiment we keep our route, but first send an ARP-request from netns22 to netnsR. This request causes a valid entry for 192.168.5.22 in netnsR’s ARP-cache for interface veth2R. Afterward we send an ARP-request for 192.168.5.22 again.

Expectation:
The valid entry for 192.168.5.22 in netnsR’s ARP-cache will be ignored. The route determines the path the ARP-requests take.

And in netnsR:

Tcpdump registers the following:

Again, we see that the entry in the ARP-cache is ignored and that the route determines the path that our pure ARP-request takes.

Experiment 9.3: ARP-requests from netnsR after switching the route

To confirm our results we now switch the route to point to veth2R and S2. Then we empty the ARP-cache in netnsR and send two ARP-requests afterward.

Expectation:
The ARP-requests will now be sent via veth2R into segment S2.

As expected. Pure ARP-requests in network namespaces with ambiguities are sent according to the relevant route.

Experiment 9.4: ARP-request for 192.168.5.22 with interface specification

We work again with “Route I” pointing to S1). But now, we specify the interface to use for both an ARP- and an ICMP-request to 192.168.5.22 in segment S2.

Expectation:
With the options for the interface in ICMP- and ARP-related commands have a meaning at all, we should see that the requests are answered if the target IP is located behind the specified network interface.

And on the ICMP side:

As a side effect we see: With ICMP-triggered ARP-communication the ARP-cache of netnsR obviously gets updated! (I checked that confirming ARP-requests from netns22 were sent later than the time of the cache update.)

We learned now that the route comes into the game, only, when the user or the program sending an ICMP request (or TCP command) does not explicitly specify an interface to use when there is a structural ambiguity due to multiple NICs.

I.e., a network namespace uses the route information when it otherwise would have to guess which interface it should choose between multiple ones for sending a pure ARP-request. This holds both for ARP and ICMP requests.

Experiment 10: Sending ARP- and ICMP-request from netnsR without routes

A final experiment: What happens when we try to send requests from netnsR, but without any defined routes? The results are:

Routes are essential!

Why do replies to ARP-requests send via userspace tools do not trigger an update of the ARP cache?

Unsolicited ARP-replies could be forged and are potentially dangerous. Such replies are used in man-in-the-middle attacks. Therefore, a kernel parameter prevents that unsolicited replies lead directly to a cache update. See here.

To see the effect we delete all entries in the ARP cache of netnsR, change the relevant kernel parameter for netnsR and issue an arping command again:

Ok, but why did the kernel regard a reply to an ARP-request sent with the help of arping as unsolicited? The answer is that a userspace-tool was used to create the request. It did not come from the kernel itself. This also explains the different reaction to ICMP-requests: The ARP-requests in respective experiments were created by the Linux kernel to support the ICMP-request.

Possible reasons for confusion

To me the results of our simple experiments appear to be completely logical and reasonable. All confusion that may have come up regarding my scenario in the last and previous posts resulted in my opinion from three points:

  • My segments did not coincide with distinguished IP-subnets.
  • Without further information there was an ambiguity in network namespaces netnsR regarding network paths, due to multiple available NICs and respective attached segments.
  • Automatically created routes led to a kind of misconfiguration on the IP-level.

As we did not specify an interface for an ICMP- or ARP-request the ambiguity in namespaces with multiple NICs was resolved by using the available, but insufficient and at least partially wrong route information.

The coupling namespace had to be correctly informed about which IP sits behind which of the NICs. In my last post this was achieved in the end by setting the required specific routes.

Conclusion

The experiments above together with experiments in a previous post show clearly:

In network namespaces (or hosts) with multiple NICs (and respective L2-segents behind) there is a structural ambiguity regarding the choice of a NIC when ICMP- or ARP-requests must be emitted.

If one does not specify the network interface in commands sending ICMP- or ARP-requests a defined route to the requests’ target IP determines which NIC is used to emit the request into a L2-segment. Such a route may be selected by the Linux-kernel among multiple defined routes according to standard selection criteria (specificity and/or costs, …). The namespace does not guess which of the available paths to choose.

Thus, a route does have an impact regarding the choice of a NIC/segment both for pure standalone ARP-requests and ARP-requests created to support ICMP and other protocols on layer 3 and above in the TCP/IP stack.

However: As soon as a ARP-request is generated in a L2-segment, a direct ARP-reply is generated by a namespace which owns a NIC with the requested IP. Such ARP-replies are not hindered by potentially conflicting routes. At least for namespaces with attached standard L2-segments. It will be interesting to see whether this remains to be true when we exchange our standard segments by VLAN-segments.

An ICMP-reply may not be sent despite available and correct ARP- and MAC-information if the route (and related interface information) is in conflict with the interface indicated by the relevant ARP-cache’s entry.

In our experiments we got no indication that ARP-requests were somehow forwarded by a network namespace between L2-segments located behind different NICs – even when the segments’ IPs belonged to the same IP-subnet. This was to be expected due to security issues.

We have also seen the ARP-cache is only updated with the information of ARP-replies if these replies are not regarded as unsolicited by the Linux kernel.

There are network configurations for which routes specified for IP-subnets are insufficient and may lead to an unwanted reaction of your network to requests. ARP-cache entries may become inconsistent with routes regarding network interfaces. Therefore, in the case of complex or unusual configurations automatically created routes (which appear in the wake of IP address assignment) have to be checked, to be corrected and/or to be supplemented.

Ambiguities in network namespaces regarding possible packet paths have to be avoided. The routes must reflect the real position of IPs in a segmented LAN. Therefore, the routes must be defined on the required detail level of the real segmentation. Again: Just referring to IP-subnets may NOT be sufficient for some peculiar network configurations. Then one may have to use routes to single, specific IPs (/32).

In the next post of this series

More fun with veth, network namespaces, VLANs – V – link two L2-segments of the same IP-subnet by a routing network namespace

we will connect our two L2-segments (with IPs of the same IP-subnet) by a correctly routing namespace with activated forwarding. We will in addition have a look at the effect of Proxy ARP in such a configuration.