This all started when I was pondering about adding a WireGuard VPN server to my router running OpenWrt and thinking the way OpenWrt is configured is kinda outdated. Having a router running GNU Guix had been on my mind for some time and said "let's give 'er a go".
There is a big opportunity to improve routing. Like most computer hardware, most routing hardware is sold running proprietary software. Consumer routers are all-in-one - chip, switch, access point like in Asus and Netgear home routers. Business router hardware use separate, individual components for ease of swapping out, support and upgrade. Common business brands are Mikrotik, Ubiquity, etc.
The part of these router systems that can be considered the "gateway", the main part driving the network, is proprietary, with very locked down hardware. The custom chipsets and boards these gateways run are unique butterflies. Butterflies which are controlled by giant corporations who love "cloud" features and obsolescence. This requires someone driven by the challenge of reverse engineering to crack into these rocks and install a free operating system. Why continue to put up with this?
I've had good experience in my life with many consumer routers. Starting with fond memories of flashing custom software to now-old Linksys routers using DD-WRT maybe around 10 years old. It was the first hardware I really started to try custom software on. Even then there was this drive to free the networked bits. Throughout the years this carried on with different router manufacturers and different operating systems. From DD-WRT to Tomato, Asuswrt-Merlin, and in recent years OpenWrt.
There is this lovely little thing called commodity hardware. Hardware of the Commons, if you will. The hardware that most computers are built with. Where it is easy to install and boot into whatever OS you want. With well developed platforms like like x86. Why continue to use the crap these companies push out when we can run on the same platform most servers on the internet use? Let's do that. The cost of commodity hardware is also much cheaper when we factor in routers being sold for hundreds of dollars with just a little bit of ram, storage, MHz. Win!
In the near-impenetrable rock realm of routers, the free operating system that is most popular is OpenWrt. OpenWrt runs BusyBox, a userland meant for systems with limited resources, which again, in the age of cheap chips and gigabyte storage, why? BusyBox was created by the same guy who wrote The Open Source Definition, an ideal in contention to the Free as in Freedom software ideal of GNU. Funny how history works out. BusyBox also runs musl libc. Well, I don't want BusyBox or musl. Give me GNU and glibc, thank YA very much.
OpenWrt has recently moved to APK, the Alpine package manager, and can basically be considered an Alpine distro at this point. I had a lot of problems with the imperative nature of this system, which I will get into.
BSD, in typical fashion, has a few corporate-y distros like OPNSense and pfSense for commodity hardware. I have no interest in these other than they are router focused distros on commodity hardware.
That feeling I had with OpenWrt where everything was much harder than it needed to be.. Configuring it is done imperatively through the CLI, through it's web interface, or through editing config files scattered about the system. Just messy. Congrats to the team for the work they do, but it is time for something better.
Those familiar with Guix already know the benefits. For those who don't you'll need to study some. Key words are declarative, reproducible, flexible; all done in a metaprogramming language - Guile Scheme. The Lisp machines of today, where the parentheses are eating and packaging everything. Your system config can be defined in one file, it's chef's kiss beautiful. We can use this same technology to define routers, making routers easier to setup and reproduce than ever before. It just takes people honing the use case.
I, like probably many, had put off digging into IPv6. This project forced me to, which is great because IPv6 is empowering. The world ran out of IPv4 addresses years ago, and this comes at a high cost today. They are getting so scarce and expensive that more and more ISPs are using "Carrier Grade" NAT to take 1 IPv4 address and share it between multiple subscribers. Thankfully my ISP doesn't do this (yet). Just imagine sharing an IP address with many other people and the problems that can cause. One of the worst is it causes people to not be able to serve ports to the internet unless some interface is created by the carrier to port forward, which is not a guarantee. This takes sovereignty away from people. IPv6 was created not long after v4 because it became obvious that v4 addresses would run out. Today, about half the internet's traffic is routed through IPv6 and half through v4.
What IPv6 gives is a virtually unlimited number of IP addresses to use. The number is 3.4 x 10³⁸ or 340 undecillion. Compare this to IPv4's 4.3 billion addresses. With this many addresses, every device can have its own IP. This means things like NAT and all the complexity that brings is not needed. Instead of receiving just 1 IPv4 address from an ISP, with IPv6 you get a block delegated of at least /64 which is 18,446,744,073,709,551,616 (18 quintillion) globally routable addresses. Think that'll do. But, and this a big but: only half of the internet's traffic is IPv6, so IPv4 must still be supported. Even today, there are a few websites from big tech companies that don't have IPv6 addresses. There are many ways to support IPv4 alongside v6, from complex 4 to 6 address translation, to the more prevalent, albeit repetitive dual-stack. Dual-stacking is running both IPv4 and IPv6 services natively, no translation, side-by-side. Native is great, the small downside is having to do things twice, serving IPs, configs, etc, must be done for each. Dual-stack is what I chose for my setup.
A few other things. IPv6 is 128 bits and written in hexadecimal. An interface can have multiple IPv6 addresses which is actually really handy. For instance you can have a link-local address, the global address (GUA), and private unique local addresses (ULA). IPv6 addresses also don't need DHCP to assign them, with SLAAC (Stateless Address Auto-configuration) this is done automatically. The smallest block SLAAC requires to work is /64. There are positives and negatives to each, DHCP is usually better suited for large networks where a lot of control is needed. And SLAAC when just keeping it simple, but with tools like mDNS you can see IPs on the network. It should be known that systems like Android don't even support DHCPv6, to the anger of many developers. There is a wealth of info there about IPv6. It will change your thoughts on networking. The mental model of many, including myself, of networking, firewalls, routing, was warped through a lens of scarcity caused by IPv4 and NAT. Took some time to wrap my head around.
I had a cheap mini-pc with a Ryzen chip in it just waiting to be used. With hardware and storage that laughs at my old router's hardware. The funny thing when I bought it new, it is half the price of the locked down, anemic consumer router I have sunsetted. Commodity. Hardware. Words to remember! Now the all-in-one routers come with switches and an access point too. I bought a 5-port switch and access point for a few dollars. Used a usb-to-ethernet adapter so I have 2 ethernet ports on the mini-pc, one for WAN and one for LAN, to the switch. Access point plugs into the switch.
Think about the combinations and possibility of hardware we can use when we aren't tied to what some company wants to sell as consumer or router gateways. Commodity hardware. Commodity hardware..
When I first started this project I tried to use a USB wireless adapter with hostapd but this was error prone. Wireless adapters are not meant to be access points, at least not good ones. Although they have the ability, in general, it likely won't be stable. Their access point mode is more temporary "hotspot" use. Wireless technology as many well know in the GNU/Linux world has always been a locked down, proprietary realm for the most part. Most the USB wireless adapters have Realtek chipsets and bad Linux drivers making for an unreliable access point mode which I experienced. The best you'll find is PCI-E Atheros cards whose drivers are much better for access point, but still not designed to solely be one. You'd also need multiple cards for dual-band wifi. If you ever wanted something like long-range outdoor access points interfaces, etc, then you'll probably be out of luck. So I am just using an external, "dumb" access point. It'd be great if things weren't this way but they are.
Linux has this fun thing where interface names aren't guaranteed to be the same due to race conditions upon boot. Udev (from the minds of systemd) slaps a rename onto them for "predictable naming".. An incomprehensible syntax based on hardware positioning, which I've read may not be guaranteed either if one were to place in new adapters into PCI slots, etc, causing a change in order of the naming scheme. Great. It'd be wise to get the MAC addresses of each interface and rename them yourself to something sane. I use wan0 for the built-in ethernet port, lan0 for the usb-to-ethernet adapter, and wlan0 for the mini-pc's internal wireless adapter which is unused.
(udev-rules-service 'static-link-names
(udev-rule
"10-static-link-names.rules" "
ACTION==\"add\", SUBSYSTEM==\"net\", ATTR{address}==\"84:45:09:11:ea:eb\", NAME=\"wan0\"
ACTION==\"add\", SUBSYSTEM==\"net\", ATTR{address}==\"8c:ae:4c:cc:33:56\", NAME=\"lan0\"
ACTION==\"add\", SUBSYSTEM==\"net\", ATTR{address}==\"5c:e4:2a:44:1e:e1\", NAME=\"wlan0\""))
There are different ways you can set up your network. I use the static-networking-service-type in Guix. Setting the LAN IPv6 (ULA) and v4 gateway addresses and DNS resolver addresses. The IPv6 address is ULA which I use for DNS purposes on my network. If my ISP had static IPs, I'd likely just use GUAs only. I'll show how to get the GUA from the ISP in a moment.
(service static-networking-service-type
(list (static-networking
(links (list (network-link
(name "lan0")
(arguments '((up . #t))))))
(addresses (list (network-address
(device "lan0")
(value "10.0.0.1/24"))
(network-address
(device "lan0")
(value "fd44:4e3a:7e85::1/64"))))
(name-servers '("127.0.0.1"
"::1")))))
A few options for DNS. Could use dnsmasq, but it isn't recursive, or could use Unbound. I chose Knot Resolver which is actually what Cloudflare uses for their resolver and powers 1.1.1.1. Running your own DNS is about increasing your sovereignty and fighting against possible censorship. For each domain the resolver will reach out the authoritative server to make sure the info is correct. And we get the benefits of setting DNS records for our local network for ease of communication, these can be set using hints in Knot Resolver which I use the IPv6 ULA of each device for. Knot resolver is configured in Lua.
(service knot-resolver-service-type
(knot-resolver-configuration
(kresd-config-file (plain-file "kresd.conf" "
net.listen({ net.lo, net.lan0 }, 53, { freebind = true })
modules = { 'hints > iterate', 'stats', 'predict' }
user('knot-resolver', 'knot-resolver')
cache.size = 100 * MB
hints.set('router fd44:4e3a:7e85::1')
hints.set('desktop fd44:4e3a:7e85:0:b7a5:9e52:3085:53c1')
"))))
I assume most ISPs are using DHCP for IPv6 delegation to your router. One exception is Starlink which uses SLAAC. Mine currently (Comcast) is using DHCP, so I use a DHCP client called dhcpcd. I set my own shepherd provision and requirement for the service so it doesn't conflict with the static networking service by creating a simple-service. The dhcpcd-service-type in Guix right now doesn't mesh well with it because it tries to provision networking itself, and there can only be one, Highlander. The code gets an IPv6 address for wan0 and assigns the block to lan0. IPv4 is obtained without further setup.
(simple-service
'dhcpcd-service shepherd-root-service-type
(list (shepherd-service
(provision '(dhcpcd))
(requirement '(networking))
(start #~(make-forkexec-constructor
(list (string-append #$dhcpcd "/sbin/dhcpcd")
"-q" "-B" "-f" #$(plain-file "dhcpcd.conf" "
nohook resolv.conf
interface wan0
ipv6rs
ia_na 1
ia_pd 2 lan0/0") "wan0")))
(stop #~(make-kill-destructor)))))
I use SLAAC for simplicity which are done through router advertisements with radvd. A simple-service is used again to make sure timing is right during boot. It advertises the IPv6 DNS, GUA and LUA subnet.
(simple-service
'radvd-service shepherd-root-service-type
(list (shepherd-service
(provision '(radvd))
(requirement '(user-processes))
(start #~(make-forkexec-constructor
(list #$(file-append radvd "/sbin/radvd")
"--config" #$(plain-file "radvd.conf" "
interface lan0 {
AdvSendAdvert on;
RDNSS fd44:4e3a:7e85::1 {};
prefix ::/64 {
AdvOnLink on;
AdvAutonomous on;
};
prefix fd44:4e3a:7e85::/64 {
AdvOnLink on;
AdvAutonomous on;
};
};"))
#:pid-file "/var/run/radvd.pid"))
(stop #~(make-kill-destructor)))))
Here we use dhcpd to assign IPv4 addresses from subnet and assign DNS. This package is actually deprecated since a few years ago for one called Kea but that isn't in Guix yet. The code in dhcpd is extremely stable and is just fine for this use.
(service dhcpd-service-type
(dhcpd-configuration
(version "4")
(interfaces '("lan0"))
(config-file (plain-file "dhcpd.conf" "
default-lease-time 3600;
max-lease-time 7200;
option domain-name-servers 10.0.0.1;
authoritative;
subnet 10.0.0.0 netmask 255.255.255.0 {
range 10.0.0.2 10.0.0.244;
option routers 10.0.0.1;
option broadcast-address 10.0.0.255;
}"))))
An important thing to really internalize is with IPv6, you have globally routable addresses. Many of us are so used to port-forwarding with IPv4 and NAT, our view on firewalls is distorted. When you have only 1 IPv4 address and any port must be explicitly forwarded, NAT is a big wall stopping traffic. But when you don't require NAT, like with IPv6, your devices are open to the world. That is unless each device has a strong firewall rules set for it - unlikely. With nftables we create rules to control the flow of packets without the need for NAT. If NAT is needed it is also done in nftables. Since my router is now powerful enough to be my server, I don't have any NAT rules currently other than masquerade needed for devices on the network accessing IPv4 resources on the web.
You are probably better off starting with the minimal config from nftables-service-type and adding each rule you need.
(service nftables-service-type
(nftables-configuration
(ruleset (plain-file "nftables.conf" "
table inet filter {
chain input {
type filter hook input priority 0;
policy drop;
# early drop of invalid connections
ct state invalid drop
# allow established/related connections
ct state {established, related} accept
# allow from loopback
iif lo accept
# drop connections to lo not coming from lo
iif != lo ip daddr 127.0.0.1/8 drop
iif != lo ip6 daddr ::1/128 drop
# allow icmp
ip protocol icmp accept
ip6 nexthdr icmpv6 accept
# allow DHCPv6 replies (UDP 547) to client (UDP 546)
udp sport 547 udp dport 546 accept
# allow lan traffic
iifname lan0 accept
# reject all else with error
reject with icmpx type port-unreachable
}
chain forward {
type filter hook forward priority 0;
policy drop;
# allow forwarded traffic from lan0 to wan0
iifname lan0 oifname wan0 accept
# allow return traffic from wan0 to lan0
ct state {established, related} accept
# allow all traffic incoming and outgoing on lan0
iifname lan0 oifname lan0 accept
}
chain output {
type filter hook output priority 0;
policy accept;
}
}
table ip nat {
chain postrouting {
type nat hook postrouting priority 100;
oifname wan0 masquerade
}
}"))))
Bootup timing is important. This may have been the hardest part of the whole project. Especially when I was first using the USB wireless adapter because it is initialized so late. You can view /var/log/messages to see what happened during boot. For my hardware, the load order with the services requiring networking, etc before starting seems to have solved any race conditions during boot.
I hope this helps those who have similar interests and progresses routers running Guix. For those who are serious about their networking and want a declarative solution running on commodity hardware will do what is needed to reap the rewards!