This is the third blog post about my home network improvements series.
In the previous post, we presented what feature should we implement in our router.
We will now see how to implement the basic features which are routing, firewall and NAT, DHCP and DNS.
- Router features list (published)
- Creating a basic router, defining the network and routing (this post)
- Adding a firewall to our router (to be published)
- Providing basic network services, DHCP and DNS (to be published)
- Extra services (to be published, could be splitted in more than one post)
So today’s post will present in order:
- OS installation
- Network interfaces configuration
- Discussion on what is routing, with activation of packet forwarding, Network Address Translation (NAT) and IP Masquerading
For some items we will see today, we will start with basic functionalities that we will improve or iterate in subsequent posts. As I have said in a previous article, I want to try out nftables instead of using iptables. But many tools I would like to use to quickly create a router are still only supporting iptables as backend, and you cannot mix iptables and nftables. Such tools include systemd-networkd, Docker, or the version of firewalld which Ubuntu is currently supporting (note that firewalld version 0.6+ does support nftables as a backend). So in this first iteration and in order to relatively quickly create a basic router, we will use mostly iptables either through systemd-networkd support or via other tools.
First step, installing the OS
In order to have a stable, not bleeding edge, long term maintained system, and one which supports both network cards out of the box, we will install Ubuntu Server 18.04 LTS. Of course, Debian 9 or openSUSE would be another good fit, but CentOS 7.5 has definitively a too old Kernel (actually it is difficult to know as Red Hat keep on backporting features from newer Kernel versions, so you never know how “modern” your Kernel is). Ubuntu with their rolling LTS enablement stack is pretty good has regularly (every 6 months) delivering a newer Kernel in a “rolling upgrade” way. The Kernel being the most interesting part of Ubuntu for running a router, this is a pretty good choice.
A rolling release distribution such as Arch Linux or Gentto Linux would have been pretty good as well. But given that I need to take care of other things than my router everyday, I tend to prefer a distribution less bleeding edge and where I just need to do a big upgrade every 2 years or so.
Other distributions such as Fedora require too much effort in upgrading every 6 or so months. I really want an LTS distribution where I could let it run (with just security updates) for many years without much maintenance.
My goal is not to teach you how to install Ubuntu Server, it is pretty easy anyway just follow the Ubuntu documentation and the text based wizard during installation. I would recommend the following though:
- Use the alternate Server installation, as I like to use LVM and LUKS (in addition the EFI boot manager worked better with my selected motherboard, with the default installation media, it rebooted successfully but after a cold start the EFI did not see any booting device, YMMV)
- If you are not using Ubuntu
then do not use the root account, create another user account with appropriate sudo rights - Use a unique password, of course not a well-known one, one you can remember and type easily (when you use sudo)
And after installation you should do a few things to harden your installation, I’m only mentioning the topics, you can look online how to do it. The following list is far from extensive and each topics can have several depth. You need to find your own trade-off between efforts to harden and the risks (security threats likelihood). Detailing those topics and providing a more thorough hardening can be another idea for an article, but not one I’m willing to write now as this would be quite some effort.
- harden SSH: use public/private keys for authentication (deny use of password); deny remote access for root user; define list of ciphers, HMAC, key exchange you accept to support; etc. Calomel offers a good documentation on how to harden OpenSSH.
- harden authentication: using pam you can configure your system to lock out an account for several seconds to a few minutes after a defined number of consecutive failed attempts. This can help mitigate against brute force attack.
- use log monitoring: simple tools like logwatch should fit the bill, but there are many solutions. The important thing is to monitor your logs and get alerts (for suspicious activity like many failed login attempt; or for problems from your application like OOM, crash, etc.).
- firewall: we will see that later as we anyway need to set-up one.
- intrusion detection: we will also potentially see that in another article in this series. But something like OSSEC or Suricata (as we will see) can be helpful.
- intrusion prevention: a simple IPS could be fail2ban, although I’m not sure how it will be compatible if we use nftables instead of iptables. We will anyway have Suricata (in another upcoming article) which can be used for that as well.
- important/security upgrade: I would also recommend to activate automatic security upgrade, using unattended-upgrade on Ubuntu, or some extension to yum/dnf or whatever equivalent you could find. I recommend it, but it can also sometimes break things, so take care when playing with that.
- audit: you might want to consider auditd and aide. But it is not easy to configure and use them in order to find the right balance between getting alerts for real problem and ignoring false alarm.
- backup: there is not much to backup, mostly it will be a few configuration files under
/etcand the Docker volumes we will use for the extra services. But backup is important as you can use them to restore clean data before they got corrupted (if you get breached). - time sync: having good time source and being synced is also important. We will see how to achieve that in another article within this series.
Configuring the Network – systemd-network
It is now important to properly configure both network interfaces. One interface shall represent what we will call the WAN (Wide Area Network) link (or the internet/public side) and the other shall represent the LAN (Local Area Network) link (or our internal/private network side).
Given that our motherboard has a free PCIe slot which we could use to add another network card (e.g. a 4 ports NIC), it is possible in the future that we use more than one port for the LAN (having a sort of switch) and more than one port for the WAN (having redundancy, using interface bonding). I haven’t played too much with bonding interfaces yet, so I will put that aside. But creating a “switch” (technically it is a Linux network bridge) is fairly easy and that’s what we are going to configure for the LAN so that all our firewall rules and what not will already use this switch (eventhough we will configure it with one port), this will allow for transparently enhancing our router with more local port for the LAN in the future.
Ubuntu 18.04 LTS, unlike its predecessors, come with Netplan for managing network interfaces. Netplan is a simple tool which allows you to write (using YaML files) your network configuration, the tools then convert your configuration to the selected backend, currently systemd-network and NetworkManager are the only supported backend. It is pretty cool as it abstract your configuration from the backend, but it is a very new project and currently only Ubuntu is using it. So it would not be that useful for people using other Linux distributions.
Ubuntu 18.04 uses systemd-network for the Netplan backend, and it is possible to directly configure this backend rather than using Netplan. That’s what we are going to do. So people using Arch Linux, Fedora Linux, Linux From Scratch, etc. can also apply the same configuration easily. The systemd-network was added to systemd with version 209 and systemd-resolve to 213, so distribution like Ubuntu 16.04 LTS should work fine with it because it shipped with systemd version 229. However, that was not my experience in the past, I had many bugs. So on Ubuntu 16.04 (or other distributions using version 229 or older), I recommend adapting the configuration to the distribution specific tools or upgrading (e.g. to Ubuntu 18.04 or a newer version of your favourite distributions).
Onto the configuration, I will use the on-board NIC (i219-V, e1000e driver, named eno1) for the WAN link. And I will add the extra NIC (i211, igb driver, named enp1s0) to the bridge which will represent the LAN link.
WAN Interface
The WAN interface should be configured by DHCP because that’s what my setup requires. It is possible that your ISP provides you with a static IP. Follow the instruction from your ISP to configure it. But you could always try the DHCP variant, it does not hurt and might simply just work. Just for the fun of it, I will rename the interface eno1 to wan1.
Let’s create a `.link` file to rename the interface to wan1. The file should be placed under `/etc/systemd/network/`. We need the MAC Address of eno1 for the `.link` file, you can get it using
$ ip link show eno1
3: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
link/ether 40:8d:5c:4c:0c:95 brd ff:ff:ff:ff:ff:ff
The address reported by link/ether (here 40:8d:5c:4c:0c:95) is the one you should use. Here is the content of my `.link` file
$ cat /etc/systemd/network/25-router-wan1.link [Match] MACAddress=40:8d:5c:4c:0c:95 [Link] Name=wan1
Note: if your ISP requires to set a specific MAC Address for the WAN link, you can override it in the above file. Just make sure to add another `MACAddress=...` (replace `...` by the MAC address your ISP is expecting) entry under the `[Link]` section.
On Debian/Ubuntu, one need to update the initramfs for the system to be able to change the network interface name. This is done that way (note that you will need to reboot to take effect):
$ sudo update-initramfs -u
After the reboot, you should have a wan1 interface instead of eno1, but you do not need to do it now and can proceed.
We will now configure the wan1 interface for DHCP. You will need to create a `.network` file that match the `wan1` interface. My ISP provide a dynamic public IPv4 via DHCP, so that’s what I’m going to configure. I’m also going to deactivate a few features on that interface which I do not want or consider unnecessary.
$ cat /etc/systemd/network/75-router-wan1.network [Match] Name=wan1 [Network] DHCP=ipv4 DNS=9.9.9.9 LinkLocalAddressing=false LLDP=false [DHCP] UseDNS=false UseNTP=false UseMTU=true SendHostname=false RouteMetric=100
If you have the luck to have IPv6, you could replace the line `DHCP=ipv4` by `DHCP=ipv6` or `DHCP=true` for both IP versions.
I have currently specified a public DNS resolver (here this is 9.9.9.9, which should be a rather good with respect to privacy DNS resolver operated by quad9). We will use it temporarily until we deploy our own DNS resolver as we will need access to some online resources and therefore a DNS comes in handy.
LAN Interface
As mentioned, we want our LAN to be defined as a switch albeit with currently only 1 port, but which could be extended to more ports if we would add a PCIe network card. We will create this switch in “software”, so we are going to use Linux bridges to implement. In systemd, we need to define a new network device, so the file will have the extension .netdev. Here is our file content:
$ cat /etc/systemd/network/10-router-lan1-bridge.netdev [NetDev] Name=lan1 Kind=bridge
The .netdev file is telling systemd to create a bridge device named lan1. We now want to add the available interface to this bridge. We can either use the device name or MAC address, this time we will use the device name:
$ cat /etc/systemd/network/25-router-wan1.link [Match] Name=enp1s0 [Network] Bridge=lan1
The LAN interface (the bridge now) will be configured with a static IP of our choosing. For this static IP, it will be one of the IP subnet we will select. We can use one of the IP subnet allowed for private IPv4 addresses (see RFC 1918). I advise to use a range within the 192.168.0.0/16 subnet to avoid collisions with your company network when working from home (many companies are using ranges in the 10.0.0.0/8 or 172.16.0.0/12) or some tools are also using such private subnets (e.g. libvirt or Docker are using ranges within the 172.16.0.0/12 by default). I would select the subnet 192.168.9.0/24. You could of course go with any others. Our router will now have the IP 192.168.9.1 (but you can give it any IP from 192.168.9.1 to 192.168.9.254, it does not really matter).
$ cat /etc/systemd/network/75-router-lan1.network [Match] Name=lan1 [Network] Address=192.168.9.1/24 LinkLocalAddressing=false LLDP=true ConfigureWithoutCarrier=yes
Nothing much to say on this configuration file, it should speak for itself. Compare to the WAN interface, I have here chosen to activate LLDP and set up the interface even when the carrier is down (not a single device on the LAN side). LLDP is the Link Layer Discovery Protocol and can provide some extra information from devices on your network which might advertise themselves. It is not mandatory to activate it, but it can give you some insights and monitoring of your local network. You can use `networkclt lldp` to retrieve information of neighbours.
Routing
We have now our “router” with two network interfaces one configured for managing our internal LAN and the other one connected to the WAN. But can it route packets from LAN to WAN and from WAN to LAN? No.
IP forwarding explained
We need to understand two aspects, first the Linux kernel will not automatically forward a network packet from one interface to the other. What that means concretely is that when a packet arrives from the LAN with a destination on the WAN, that packet is received on the “network stack” on the LAN interface.
You can imagine a “network stack” as a bucket of water, currently we have 2 buckets, one on the LAN side and one on the WAN side. Each bucket as a water pipe attached to it representing respectively the LAN and the WAN networks. So when a device on the LAN push a small quantity of water, it arrives in the bucket of the router LAN interface. Now someone has to take a cup, take the water from the LAN bucket and put it in the WAN bucket so it can be distributed on the WAN network. That someone is your kernel and you need to tell it to do so, it is called IP forwarding and can be configured using sysctl entries (or the /proc filesystem) on a per interface basis or for all, or you can use systemd IPForward option.
So we have learnt that one job of the router is to pick up requests from the LAN interface and put them (or forward them) on the WAN interface to the next hop or to the final destination. By default the forwarding of packets from one interface to another is disabled, so we must instruct the Kernel to do so. As an example, the site berthon.eu has for IP address 213.186.33.87 so my laptop when opening that site will create a network packet at destination 213.186.33.87 (at internet layer or layer 3), as the IP is not part of a subnet my laptop belongs to, and because I have no special routes defined for that destination, my laptop will transfer this packet to the default route which is my router. It will not change the destination IP from 213.186.33.87 to 192.168.9.1, it will request (if not already cached) the MAC Address (Link layer or layer 2) for the IP 192.168.9.1 so the lan1 interface. From the layer 2 perspective this is just a network message going from my laptop to the router, but from the layer 3 perspective the message is en-route towards berthon.eu. At our next hop, the router will receive the packet on the lan1 interface. Now what should happen is that the router will detect that the destination IP is not his own and that it should forward the packet. Checking the router own routing table it will detect that it needs to forward the packet on the wan1 interface and send it to the next hop until it reaches berthon.eu.
IP Masquerading, Network Address Translation
That’s all good, but what happens when the other end receive our message and wants to reply. In our case now, it will receive the message or packet with a source IP which is the one on our LAN, so a private IP subnet. The remote end, in our example berthon.eu, has no means to send back an answer as private IP subnet should not be routable from a public IP address. It’s understandable, there might be several organisations or home networks which are sharing the same private IP range and perhaps the same private IP addresses. So those addresses are not unique and cannot be routed from the public internet (and should not be routed for security reasons). That’s where we need Network Address Translation and more particularly IP Masquerading, that’s our second aspect. What it means for our current use case, our router needs to masquerade the internal LAN IP address using its external WAN IP address.
A fun experiment
Let us do a concrete example. We will use the traceroute utility, on Ubuntu 2 packages provide this utility with different level of capabilities, I tend to prefer the utility for the eponym package, i.-e. traceroute, but you could also use the one from inetutils-traceroute. I will use a subset of options which are common to both utility so you might not need to install one. We will perform a route discovery using this utility and the ICMP protocol towards the Quad9 server at IP address 9.9.9.9. First, when running this command on my laptop which is connected to the existing network using my ISP-provided router, I get the following output:
$ sudo traceroute -I -q1 -w1 -n 9.9.9.9 traceroute to 9.9.9.9 (9.9.9.9), 30 hops max, 60 byte packets 1 192.168.1.1 0.721 ms [...] 15 9.9.9.9 28.923 ms
We can successfully trace the complete route until the Quad9 server, it requires 15 hops, the first one being my ISP-provided router, the 15 hop being the Quad9 server.
Now let’s try to use our new router and repeat the above traceroute via it. For this, we will simply plug the WAN port of our new router to a free port on the ISP-provided router (so the family does not complain that there is no internet connectivity and as our new router is not yet secured it is better to stay behind your ISP one). Our router should now get a valid IP address on the WAN interface and be able to connect to internet, mine got the IP 192.168.1.12. You can test connectivity by doing ping -c2 9.9.9.9 on your new router. We do not have DHCP or DNS yet, so we will need to plug a test machine to the LAN interface and configure it using a fixed IP address. I’m using my laptop again, I am plugging the ethernet port on my laptop to the LAN port on the new router. Then on my laptop I have configured a fixed IP 192.168.9.2 and setup the default router to be 192.168.9.1. When I re-execute after that the traceroute command I get the following output:
$ sudo traceroute -I -q1 -w1 -n 9.9.9.9 traceroute to 9.9.9.9 (9.9.9.9), 30 hops max, 60 byte packets 1 192.168.9.1 0.628 ms 2 * 3 * [...] 29 * 30 *
The command fails to get replies (except for the first hop) and after 30 hops stops. Our new router does not seems to route, which after you have read the above is expected. But let’s see what is exactly happening on the network. We should open 2 extra terminals (or screens), so that we can run and see 3 different commands in parallel. For clarity, we will name in this chapter terminal 1 “laptop”, terminal 2 “router-lan” and terminal 3 “router-wan”. So on the router-lan and router-wan terminals, you first need to connect via SSH to the new set-up router on each of those terminals. Once connected run the following commands:
router-lan $ sudo tcpdump -n -i lan1 '(icmp[icmptype] == icmp-echo or icmp[icmptype] == icmp-echoreply) and host 9.9.9.9'
router-wan $ sudo tcpdump -n -i wan1 '(icmp[icmptype] == icmp-echo or icmp[icmptype] == icmp-echoreply) and host 9.9.9.9'
Now repeat the traceroute command inside the laptop terminal and we will see what is displayed on the other 2 terminals. On the router-lan terminal, we can see 30 lines almost identical (except for the time and sequence number), something similar to “23:47:11.088759 IP 192.168.9.2 > 9.9.9.9: ICMP echo request, id 15899, seq 1, length 40” which the the echo request being received on the LAN interface. Nothing is visible on the WAN interface. We are not routing packets.
Now on router-lan, interrupt the tcpdump command by pressing Ctrl+C. On that terminal we will change the IP Forward Linux kernel setting for the lan1 interface only. Execute the following command:
router-lan $ sudo sysctl -w net.ipv4.conf.lan1.forwarding=1 router-lan $ sudo tcpdump -n -i lan1 '(icmp[icmptype] == icmp-echo or icmp[icmptype] == icmp-echoreply) and host 9.9.9.9'
One more time, we execute on the laptop terminal the traceroute command and check the output on the other terminals. Now we can see the same packets on both lan1 and wan1 interfaces, we are routing packets to the outside. But on both interface we still only see “ICMP echo request” packet types, no “ICMP echo reply” one. That’s when we need to activate IP masquerading. As you can see in the output of router-wan terminal, the packet has for source IP 192.168.9.2. When the Quad9 server will receive this ICMP echo, it won’t be able to answer it and will probably drop the packet (actually given that we still have the ISP router which more than probably does IP masquerading, the packet which will reach the Quad9 server has probably as source IP your ISP router public IP, so it is able to send back the packet, but the ISP router is then not able to route the packet back to 192.168.9.2 because it is an unknown subnet. If I could add a new route inside the ISP router I would not need to do IP masquerading on the new router). So we need to tell our router to replace the source IP by its own IP from the wan1 interface. For this test, we will use iptables. Note that the following change as well as the previous one are non-permanent changes which we can use until next reboot. So after this experiment a clean reboot will reset all our changes. On the router-lan terminal, interrupt the running command and run:
router-lan $ sudo iptables -t nat -A POSTROUTING -o wan1 -j MASQUERADE router-lan $ sudo tcpdump -n -i lan1 '(icmp[icmptype] == icmp-echo or icmp[icmptype] == icmp-echoreply) and host 9.9.9.9'
We run again our traceroute command and it again fails, which is to be expected because we did not yet instruct the kernel to forward packets from wan1 to lan1. But this illustrate another nice feature. On laptop terminal, we see like the 2 last attempts the same output with no reach to the Quad9 server. On the router-lan terminal, there is also no change compare to the previous 2 runs, only the date as change, but we get 30 lines of “ICMP echo request” and no reply. However things have changed on the router-wan output. First we can see that the masquerading is really happening, now packets are displayed this way: “00:17:05.222098 IP 192.168.1.12 > 9.9.9.9: ICMP echo request, id 19646, seq 2, length 40” hooray, the masquerade is working the source IP address is now the one from the wan1 interface. But we see another thing on this terminal, we now see “ICMP echo reply” packets like “00:17:05.251588 IP 9.9.9.9 > 192.168.1.12: ICMP echo reply, id 19646, seq 16, length 40“, hooray again!!! What we see here is that our packet went from the laptop, arrived at lan1, got forwarded to wan1 and routed to 9.9.9.9 masquerading the source IP address with the wan1 one, 9.9.9.9 replied and that reply has now reached wan1 back. We simply need to activate IP forwarding on wan1 and it will now work. So one last time, interrupt the command on router-lan terminal and type:
router-lan $ sudo sysctl -w net.ipv4.conf.wan1.forwarding=1 router-lan $ sudo tcpdump -n -i lan1 '(icmp[icmptype] == icmp-echo or icmp[icmptype] == icmp-echoreply) and host 9.9.9.9'
And this time it worked. The traceroute displays:
laptop $ sudo traceroute -I -q1 -w1 -n 9.9.9.9 traceroute to 9.9.9.9 (9.9.9.9), 30 hops max, 60 byte packets 1 192.168.9.1 0.597 ms 2 192.168.1.1 5.146 ms [...] 16 9.9.9.9 29.277 ms
The difference with our first run is that we have 1 extra hop. The first hop is now our new router, the second hop is still the ISP-provided router and the Quad9 server is now hop 16th.
So this is how routing, IP forwarding and IP masquerading are working together to provide a router functionality. We will now reboot the new router to clean-up the above changes. You need to reconfigure your laptop to the previous network configuration (before you changed it to fixed IP). You need to change the network cables to restore the previous state. And now we will see how e can configure the above with systemd-netword.
IP Forwarding and Masquerading configuration
We need to configure IP Forwarding for the LAN and WAN interface, and IP Masquerading. We will use the systemd setting here, it has the drawback that it enables forwarding for all interfaces and not limited to the interface we configure and that it uses iptables instead of nftables as a backend, but it is very convenient and easy to setup. So for this first router configuration, we will go for systemd. In another blog post when we will be using nftables, we will remove this configuration.
Simply edit the file /etc/systemd/network/75-router-lan1.network and add IPForward=ipv4. Usually possible values are no, yes, ipv4 or ipv6 which stands respectively for no forwarding, forwarding of both IPv4 and IPv6, forwarding of IPv4 only or forwarding of IPv6 only. We also need to configure IP masquerading on this interface, this can be set by adding IPMasquerade=yes, it is a bit weird to do that on the LAN in my opinion and I made first the mistake to do it on the WAN, but that’s the way systemd-networkd is working.
$ cat /etc/systemd/network/75-router-lan1.network [Match] Name=lan1 [Network] Address=192.168.9.1/24 LinkLocalAddressing=false LLDP=true IPForward=ipv4 IPMasquerade=yes
For the WAN interface, it is similar, the file is /etc/systemd/network/75-router-wan1.network.
$ cat /etc/systemd/network/75-router-wan1.network [Match] Name=wan1 [Network] DHCP=ipv4 DNS=9.9.9.9 LinkLocalAddressing=false LLDP=false IPForward=ipv4 [DHCP] UseDNS=false UseNTP=false UseMTU=true SendHostname=false RouteMetric=100
To activate the change simply do sudo systemctl restart systemd-networkd.service. We can now verify that the changes are activated:
$ grep -H '' /proc/sys/net/ipv4/conf/*/forwarding /proc/sys/net/ipv4/conf/all/forwarding:1 /proc/sys/net/ipv4/conf/default/forwarding:1 /proc/sys/net/ipv4/conf/enp1s0/forwarding:1 /proc/sys/net/ipv4/conf/lan1/forwarding:1 /proc/sys/net/ipv4/conf/lo/forwarding:1 /proc/sys/net/ipv4/conf/wan1/forwarding:1 /proc/sys/net/ipv4/conf/wlo1/forwarding:1 $ sudo iptables -t nat -L -n Chain PREROUTING (policy ACCEPT) target prot opt source destination Chain INPUT (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination Chain POSTROUTING (policy ACCEPT) target prot opt source destination MASQUERADE all -- 192.168.9.0/24 0.0.0.0/0
We can see from the output that all interfaces are the forwarding option set to 1, which means that IP forwarding is enable on these interfaces. The second command is displaying the firewall rules (iptables-based) of the “nat” table and we can see that in the POSTROUTING chain IP masquerading has been setup for all IP address from the LAN interface towards any other subnets (including the WAN or internet).
Routing is now configured, let’s create a very simple firewall (next post).
Footnotes
All product names, trademarks and registered trademarks are property of their respective owners.
Image credits:
- Gateway Appliance – License CC BY-SA by Cuda-mwolfe
- Ubuntu Logo – Copyright Canonical
- Network Configuration – LGPL3 Oxygen Icon by KDE
- Chicago Transit Authority Switch Tower 18 (Loop Junction Chicago) – CC BY 2007 by Daniel Schwen
- Others are credited Vera & Jean-Christophe Magical-World.eu CC BY-SA
