RC06: Notes on creating a TUN interface
I'm attending the Fall 1 batch at Recurse Center! Posts in this series cover things I'm working on or find interesting during my time here.
While working on
tcpip, one of the first things I did was set up a TUN interface, a virtual network device operating at the Network Layer/Layer 3 of the network stack. I used the following commands:
sudo ip tuntap add dev tun0 mode tun user $USER sudo ip addr add 192.0.2.1 peer 192.0.2.2 dev tun0 sudo ip link set dev tun0 up
I ran these commands with only a vague understanding of what's going on - it creates a TUN device called
tun0, assigns it two IP addresses for some reason and sets the interface to
However, I had questions, which I kept ignoring as I was making progress in terms of code. When I got stuck on another part of the project and was unable to code, I decided figure out what these commands really do. This post is a collection of notes I made along the way.
Why are there two addresses for a TUN interface? #
The first thing that confused me was the mention of two addresses. I initially thought that it might not be significant, however any packets I created used the peer address as the source address,
192.0.2.2, which didn't make sense.
I started with the
man page for
peer ADDRESS the address of the remote endpoint for pointopoint inter‐ faces. ...
Well, this just threw more terms I don't understand at me, particularly
pointopoint interfaces. So I focused on understanding that first, with the help of an answer on StackOverflow.
A point-to-point interface connects two hosts directly. The only hosts on this interface are
192.0.2.2 in our example.
Due to this, this interface does not have a MAC address and anything related to Layer 2, like ARP or the Address Resolution Protocol. This is in contrast to an Ethernet device, like the
eth0 interface, which is not a point to point interface and has a MAC address.
Next came the
remote endpoint part of the explanation. Point-to-point interfaces were still not clicking at this point, so I was confused on why the remote address was even required.
This confusion was cleared by a beautiful answer on Stack Exchange. I didn't even try paraphrasing this because it's explained so well:
In ancient times, when people used to connect a modem device to the telephone line and dial the phone of an internet provider to establish connection to the internet, the
pppddaemon used to be responsible to establish a point-to-point tunnel to the server located at the other end of the call. On those tunnels, the local address was the address assigned to the network interface being created in kernel network stack and the remote address was the local address of the other computer answering the phone call and, also, the address set as the default gateway in the local side.
For TUN virtual devices, your user-space program will act as the remote computer at the other side of the tunnel. Therefore, in order to have your program injecting IP packets to the kernel network stack, the program is supposed to generate packets with source address set to the tunnel remote address (
192.168.69.1) and receive packets whose destination is set to the same address.
After reading this, it all clicked into place.
Why is the state DOWN when I set the link to UP? #
This confusion comes from the output of the TUN interface settings.
$ ip addr show tun0 8: tun0: <NO-CARRIER,POINTOPOINT,MULTICAST,NOARP,UP> mtu 1500 qdisc fq_codel state DOWN group default qlen 500 link/none inet 192.0.2.1 peer 192.0.2.2/32 scope global tun0 valid_lft forever preferred_lft forever
This situation appears when the link is set to up, but is not being actively used by a program, indicated by the
NO_CARRIER flag. This seems similar to a laptop charger being connected to a socket, but the charger isn't plugged into a laptop yet.
tcpip contains code that connects to this interface. So once the program is run, the connection is established, and the interface's output changes:
$ ip addr show tun0 8: tun0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 500 link/none inet 192.0.2.1 peer 192.0.2.2/32 scope global tun0 valid_lft forever preferred_lft forever inet6 fe80::2db0:db32:1898:51a5/64 scope link stable-privacy proto kernel_ll valid_lft forever preferred_lft forever
LOWER_UP appears and the state changes to
Why does pinging work differently for the two addresses? #
Another useful resource I found was a post titled Tun/Tap interface tutorial. It goes in-depth about how TUN interfaces work and are set up.
The part that confused me was the section about pinging the two addresses (
10.0.0.2 in the article), and the difference in output. The interface address (
10.0.0.1) responded to pings, whereas the peer address (
10.0.0.2) did not, and I didn't fully understand the explanation provided.
I also wanted to know where the ping packets for the interface address were going, which wasn't specified in the post. So I decided to capture ICMP packets at all interfaces to find out.
First, I ran
openvpn in the background to keep the TUN interface up, which I discovered from the tutorial post.
$ sudo openvpn --dev tun0 2023-10-22 18:21:54 OpenVPN 2.6.1 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] [DCO] 2023-10-22 18:21:54 library versions: OpenSSL 3.0.8 7 Feb 2023, LZO 2.10 2023-10-22 18:21:54 TUN/TAP device tun0 opened 2023-10-22 18:21:54 Could not determine IPv4/IPv6 protocol. Using AF_INET 2023-10-22 18:21:54 UDPv4 link local (bound): [AF_INET][undef]:1194 2023-10-22 18:21:54 UDPv4 link remote: [AF_UNSPEC] ^Z + Stopped sudo openvpn --dev tun0 $ bg + sudo openvpn --dev tun0 &
This command is also used to create the interface, but in this case works as the program connecting to the interface. Then, I started capturing packets with
$ sudo tcpdump -ni any 'icmp' listening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144 bytes
ping -c 1 192.0.2.1 from another shell session, which returned the following packets (emphasis mine):
18:22:35.053952 lo In IP 192.0.2.1 > 192.0.2.1: ICMP echo request, id 28435, seq 1, length 64 18:22:35.053964 lo In IP 192.0.2.1 > 192.0.2.1: ICMP echo reply, id 28435, seq 1, length 64
The packets came from the loopback interface (
lo) and the source and destination addresses are the same, interesting.
ping -c 1 192.0.2.2 returned only one packet (emphasis mine):
18:22:37.419702 tun0 Out IP 192.0.2.1 > 192.0.2.2: ICMP echo request, id 6389, seq 1, length 64
The source address is
192.0.2.1, and the packet is captured at the
tun0 interface. Also, no echo reply packet after the request.
Why does each address return different packets? I think it may have something to do with the routes created when the interface is set up.
Checking the routing table #
$ ip route show table all | grep tun0 192.0.2.2 dev tun0 proto kernel scope link src 192.0.2.1 local 192.0.2.1 dev tun0 table local proto kernel scope host src 192.0.2.1
Once again, lots of words and little to no idea of what they mean. A Recurser pointed to this guide, which helped make sense of the terminology.
Let's start with the route for
192.0.2.2, the peer address:
192.0.2.2 the route is for the address 192.0.2.2 dev tun0 reachable from interface tun0 proto kernel route added by the kernel when setting up the interface scope link valid only on the mentioned interface, tun0 src 192.0.2.1 preferred source address when sending to this destination
src part was a little confusing at first, however it does match the ping output for
192.0.2.2 mentioned earlier. The source address there was
192.0.2.1, and I think its because the peer address is reachable only via the local address, being a point-to-point link.
The reason there is no echo reply from
192.0.2.2 is because the connected program has no implementation for ICMP or the TCP/IP stack in general. It doesn't use the kernel's network stack and requires its own, which is where
tcpip steps in.
Next comes the route for
192.0.2.1, which looks pretty different:
local 192.0.2.1 address 192.0.2.1 is locally hosted on this machine dev tun0 reachable from interface tun0 table local part of the local routing table proto kernel route added by the kernel when setting up the interface scope host valid only on this machine src 192.0.2.1 preferred source address when sending to this destination
This is specified as a local address, an address directly accessible to the machine. The route is added to a routing table called
local. When pinging local addresses, the packets are sent in and out from the loopback interface,
lo, as seen earlier.
Being a local address, the kernel's network stack manages the sending and receiving, which is why there was an echo reply when pinging to this address.
This is how local routes for other interfaces are written too, which I found out from this comment from the tutorial post.
More questions? #
TUN interfaces are starting to make a lot more sense now, but it also raises one more question. Both the hosts are only accessible within this machine. I'd like to send TCP packets to other hosts on the Internet, which doesn't seem possible with this setup.
Turns out it is possible, with the help this of NAT or Network Address Translation, which I'll save for another post as this one has a TUN of information already. 😅