Home network improvements

Currently my home network is pretty simple … at least for a computer scientist!

My ISP provided an all-in-one box with TV, landline and network router. The latter being very limited and with a crap WiFi access point (AP). So I’ve been using my old Asus RT-AC68U router as a gateway, a 24 ports switch and a Ubiquiti Unifi AP for providing WiFi in the complete house (and garden). The router and switch went into the basement whereas I’ve placed the AP roughly in the house centre. The ISP box could not be configured as bridge but supported to set a DMZ host, so I’ve configure the Asus router to be the DMZ.

Here is the basic setup:

+--------+             +--------+
|        |    DMZ      |        |          +------------------------+
|ISP Box +-------------+ Router +----------+ Switch                 |
|        |             |        |          +--+------+---+---+---+--+
+--------+             +--------+             |      |   |   |   |
                                              |      |   |   |   |
                                           +--+--+   +   +   +   +
                                           | AP  | Home Network / Lab
                                           +-----+

So I’m using only 2 ports on my router (or more exactly network gateway), the WAN and one on the LAN. This router is the peace in my current network I want to change and I will explain why and how.

Post updated on 2018-06-13.

Upgrading it

Why upgrading a running system? First of all, I started to be unhappy with the features of my Asus router as it did not support functions I wanted to use such as VLAN, DNSSEC; more advanced DHCP functions; newer Linux Kernel; etc. Some of these functions are possible by flashing the router and using an alternate firmware such as the one from Merlin (aka AsusWRT-Merlin) but it was simpler to turn off DNS and DHCP on this router and use a Raspberry Pi for the job and Docker to deploy those services (which is what I did).

In addition, I wanted to use some more security features such as an IPS (Intrusion Prevention System) and the possibility to activate on-demand “spying” capabilities to see what is currently happening on the network (e.g. my wife was complaining her phone battery was depleting faster, when in fact an application was just a bit too verbose on the network), of course consent of the household inhabitants was requested :-). Asus is providing such features (more or less detailed(advanced), but I do not like that it requires for most to accept being spied by a 3rd party. These means for deeper inspection of traffic should only be used when absolutely required and within the trust we share within the family, no other 3rd party should be involved IMHO. In addition, given our ISP bandwidth can be up to 400Mb/s it proved too demanding for the old Asus RT-AC68U hardware.

Secondly, I got very disappointed by Asus which quietly set settings that I consider sensitive relative to privacy and/or security. In one of the recent update (or perhaps because I decided to try their mobile App), Asus activated without consent DDNS (Dynamic Domain Name Server) a feature which allow to uniquely name your router and map it to your IP address. One use is for accessing a network service exposed by your router without having to remember your IP address (especially if it’s changing often). Sadly that’s a breach of the EU GDPR, because IP address are considered personal data Asus should have ask for consent between gathering this information for a service I do not want or required. EU GDPR is applicable for more than 2 years (even if only enforcable since a few days). And I do not like my IP address being tracked all the time. And then Asus also re-activated a feature I had explicitely deactivated which is to provide the Admin interface on the WAN interface, so basically allowing anyone to remotely try to connect to my router and if they brute force the credentials they could compromise my network. It is a huge security risk and given the number of people not changing credentials and the number of IoT or consumer network appliances which have made the news in the past 2 years regarding botnet, DDoS, and other global major hacks, this is plain irresponsibility from Asus to force enabled this feature without consent! So I had enough with Asus router.

I do have to be honest with Asus which still supports this particular router, adding new features and correcting bugs/vulnerabilities, close to 4 years after having purchased it. Of course given they are stuck with a Kernel 2.6.36 (the problem of many ARM SOC based platforms made by Broadcom, Qualcomm and the sort, between the platform design and being shipped to end customers, there can be years and they do no provide Kernel update!) they also fix what they can. And compare to other brands of consumer network appliances Asus tend to fair much better with respect to security/long support. But that does not excuse the mistakes I mentioned. One more thing, the latest Asus firmware upgrade (published just a few days while I was writing this article) is offering better control of what data you want to share with Asus and/or 3rd Parties (getting compliant with EU GDPR). However, when you are privacy savvy and disable the sharing of your own personal data, the extra features are disabled.

Therefore I needed a more capable network gateway and firewall in terms of raw power, network features and security/privacy. I also wanted to run Unbound as a recursive DNS resolver (with support for DNSSEC), Suricata or Snort for IPS, etc.

Which router?

I’ve looked on the market for more advanced router. Most consumer oriented routers do not provide more features than what Asus is providing. Higher priced routers tend to have more marketing WiFi powers and what not. But they are all quite limited in terms of hardware and “advanced” network features.

I’ve evaluated COTS solutions such as Ubiquiti Unifi Security Gateway but eventhough they are quite close to what I would like in terms of features, their DNS/DHCP capabilities are quite limited and I’ve found the pricing to power ratio not too good. In addition, I do not know Ubiquiti enough to be able to evaluate how long they would support this hardware.

The Czech CZ NIC organisation is making the great Turris Omnia router. It is running the Turris OS, a fork of OpenWRT. So it is a very capable, quite powerful and feature-ful device. Why didn’t I chose it? I’ve found the User Interface (UI) not so practical and easy, I have been considering using the command line which in some cases would have been easier to use than the Web Interface. Firewall rules would be more complicated to write in the Web UI than I would have using iptables (or even better firewalld). The UI suffered from some limitations, example the DHCP local domain can only be a “TLD” (e.g. .local) but cannot be a full sub-domain (e.g. .home.berthon.eu). Turris OS uses Knot as a DNS resolver, and it is a very capable and performant one, but by default DHCP leases are not communicated to Knot so there is no DNS resolving for the local LAN. This is all possible to overcome by using the command line, but then you break the UI.

So I was not satisfied and I thought I would be better off building my own router.

Building your own router – part 1

My journey started with the goal of doing as little as possible with respect to building my own router. In a second part (to be published), you will see that I fell far away from that goal.

There are 2 aspects of a router, the software (including Operating System, user interfaces to configure it, etc.) and the hardware (what physical capabilities your router will have). My first approach was to pick up a pre-package solutions for each aspects rather than doing both from scratch. This would save me time and effort and also it is always safer because you can rely on the many eyes and experiences of the people that have build those parts.

Choice of OS and Software Stack

So I decided I would use a router-oriented distribution such as Linux-based IPCop which I discovered was no longer maintained, then I went to the FreeBSD-based monowall which is also discontinued… Indeed the last time I’ve used such solution was around 2002-2003… So I had to look to something still supported and there are quite a lot of solutions between of course DD-WRT and OpenWRT/LEDE, IPFire and VyOS for linux-based solutions and OPNsense and pfSense for FreeBSD-based ones. Of course there are more solutions. I decided to go with OPNsense as the WebUI seems clear enough, the team behind seems to be active and provides regular updates. The big draw back (as for the other solutions) is that it would be difficult to automate from remote using a solution like Ansible. I prefer to backup code using Git than relying on binary blob backup, so I would prefer to do most of the configuration using Ansible (and code under Git) and only use the Web UI for monitoring, visualisating, etc. But there is no such solution, so OPNsense seems the best next bet.

Choice of Hardware

And similarly to the OS, I wanted a turnkey solution (or as close as possible). So I went first looking at OPNsense hardware offering (from Deciso) but with prices starting at 400€ for an AMD A10 dual core (Jaguar 1st Gen) with 4GB RAM, that was a bit pricey. Then I remembered that in this range of power there was the Fitlet from Compulab (with a stronger A10 CPU aimed at mobile instead of embed) and the APU2 platform from PC Engine (Jaguar 2nd Gen, quad core). But all 3 solutions are based on CPUs released around 2013 (5 years old) and they would not cope with gigabit routing and IPS. In addition, they weren’t really much cheaper than OPNsense solution.

But Compulab had a newer Fitlet model called Fitlet 2 based on Intel Apollo Lake, so quite recent. In price, it’s about as much as Deciso (when adding the facet, RAM, etc.), but the hardware is much more capable. Slightly cheaper are Zotac CI 327 and Shuttle DS77U but they are limited to 2 network interface cards (NIC), no upgrade path (e.g. no PCIe extension). But given the Meltdown vulnerability affecting Intel and the need to use PTI for mitigation with quite a performance penalty for workload doing lots of kernel/user space switching (which I think an IPS might be doing), I wanted an AMD CPU or an Intel CPU with PCID capabilities which sadly Intel did not implement in its “Atom” line of CPUs which the Apollo Lake family belong to. So given the price of a Core i3 (and no I will not buy Qotom, price is too low to be true and they even advertise to help you avoiding paying tax or software licenses) for example the Zotac CI 527 starts are slightly more than 300 € barebones (so it quickly is more than 400 € when adding RAM and SDD), I went looking towards AMD Raven Ridge (Zen-based APU) for a modern, fast and energy efficient CPU. Sadly I did not find any solutions with a low power Raven Ridge (max TDP 35W) and with at least 2 NICs.

In the end I took the decision to really build my own hardware from parts and to base it on an Intel Pentium Gold processor because they are low powered (max 35W TDP), based on the Core micro-architecture (like a Core i3, so they have PCID) and they are about 60€ (+10€ for a cooler) and can go up to 3GHz. I could find a motherboard with 2 embedded (non-Realtek) NICs so it would be easy to build a router based on these 2 core parts.

Hardware parts

I decided to select parts in order to fulfill the following requirements: high throughput; low latency; and low consumption. One should notice that those criteria are competing. For example lowest consumption can be achieved but at the expense of higher latencies, etc. So I wanted to select a CPU ideally at 15W TDP (max 35W TDP would still be OK), with 1 or cores and no hyper-threading (or that it can be deactivated). Why only 2 core? Having more cores might increase latency (unless one use process stickiness, but it is a lot of configuration) and usually in the same budget and TDP envelope a CPU with less core achieves higher frequencies (so more instructions per seconds) therefore higher throughput. One ideal platform would be embedded or mobile platform, they usually have low powered but still capable CPUs, albeit usually soldered to the motherboard. Sadly I did not find one “all-in-one” board with such CPU and already 2 NICs, or at least 1 or 2 PCIe port to host more NICs. That’s why I have selected lower powered desktop system parts:

• CPU: Intel Pentium Gold G5400T
• Cooling: Artic Alpine 11 Passive
• Motherboard: Gigabyte B360N WiFi
• RAM: 2x4GB DDR4
• SSD: WD Green 120GB
• PSU: Be Quiet! System Power B9 300W
• Case: Sharkoon QB One

Total cost: 355 €.

The CPU is an Intel Core-based processor with a 35W TDP and 3.1GHz max frequency with 2 cores and hyper-threading. The maximum consumption is bit high but I do not expect to reach it often as the graphical core will be seldom used and I do not expect to have both cores at 100% most of the time. It allows me to confidently use a passive cooler.

The motherboard is for desktop, so I do not expect it to be very low power, but it is CEC 2019 Ready (a California standard for low consumption, this option can be activated in the UEFI setup) and not “gaming” so I expect it to be decently low power for a desktop motherboard. It has 2 network interfaces, both Intel based, and a free PCIe 3.0 16x free slot. So I could upgrade this installation to support either 10Gb/s or adding more 1Gb/s NICs.

For the RAM and SSD, they are both way too much. But I did not find smaller parts for DDR4 RAM and they are a few smaller SSDs (32 or 64GB) but that would have saved just a few €, there is only about 3€ difference between the cheapest 32GB SSD and the cheapest 120GB SSD. Not worth sparing money here. I’ve selected the WD Green because its specification mentioned very low power consumption in idle and active. It is not very fast, but this is not a concern for a router.

Nothing to say with the PSU, apart that I could possibly have chosen a non-ATX one with lower max power, but ATX are cheap. As for the case, it is a bit bigger than I wanted/needed, there I could have chosen something different.

Assembling all the elements was pretty easy, it is not a hardware blog, so I won’t detail it here, but feel free to contact me if you have problems assembling hardware.

Update 20180613: The power consumption of this solution (measured on the wall plug) is about 20W when idle with a stock Ubuntu 18.04 LTS Server installed. With some tweaks in EFI Setup and OS, one can spare another 2-2,5W (and maybe more), I will details those tweaks in subsequent posts. I’m a bit disappointed as I would have expected something much lower, and was even dreaming this could be below 10W. My assumption is that the motherboard, although CEC 2019 ready, is not really low powered optimized. I also need to make sure the WiFi/Bluetooth chipset is really deactivated, perhaps I can skim a few more watts. But my expectation now is that I won’t ever be under 15W. IMHO 20W is too much power for an always-on device which will act as a router. Hardware can certainly be improved!

Conclusion

Finding a new router was very time consuming. There are really no satisfactory COTS and the best alternative when you care about networking is to build your own router from standard PC parts.

After assembling the router, I installed OPNsense which is super easy to install, follow their well explained guide if you need help. However, nothing is super easy and especially fast in computer science. I took a calculated risk in choosing a rather recent CPU and platform, but knowing I did not need the graphic part, and that both i211 and i219 NIC are well supported on FreeBSD (and thus OPNsense), I expected to be on the safe side. Sadly the i219 part is not (yet) recognised by OPNsense (or FreeBSD). It seems that Intel on their new B360 chipset decided to tweak something in the i219 part which made it not recognisable by the em driver from FreeBSD. That’s where I also discovered that when something on FreeBSD does not work, I’m less comfortable investigating it than I would be on Linux, simply because I’m quite knowledgeable in the latter and still discovering the former.

It is just a matter of time before FreeBSD will eventually support the “updated” chipset and that this change lands also in OPNsense. But I want to finish building my own router, so I decided to take a longer road: installing Linux and using nftables to configure firewall and NAT. Why not using traditional netfilters (aka iptables and all)? If I take the challenge to build my own router OS, I also want to try new stuff. Why Linux? It is just that I’m more familiar with it than FreeBSD, so for something critical like a router it is a safe choice for me, and running a recent Linux on my self-built router showed no sign of incompatibilities with the network cards. Check back soon for a new post on how to build a router on top of Linux.

We judge ourselves by what we feel capable of doing while others judge us by what we have already done. – Henry Wadsworth Longfellow

Finally, if you were just someone looking for a router and stumbled on this article, either you are a Linux/FreeBSD or network expert and I advise you to try building your own router, perhaps pick slightly older parts so that you can run OPNsense on it or wait a few months for the support to come to OPNsense. But if you are no network or Linux/FreeBSD expert, then consider the traditional network appliance vendors, or if you are a bit daring try Ubiquiti’S USG or CZ.nic’s Turris Omnia.