Last update: 2024-06-30
Motivation
Cloud services are ubiquitous these days and even sensitive material can be passed through those with appropriate encryption. However, in my home, I want stable solutions that are not susceptible to bankcrupties, buyouts, or connectivity/remote server issues. I want to have a local-first and customizable network that allows me to:
- Have stable 2.4/5GHz WLAN and 1Gbit LAN connectivity, with customizable routing/firewall tables, DHCP and a local DNS cache
- Sync files between devices at LAN speeds, primarily Org and offline music, and offloading photos from my phone
- Handle Internet of Things devices locally (via Home Assistant); allow isolating them from internet
- Run a central radio
With IPv4 still dominating internet traffic, each household usually has a NAT already, so basic connectivity is often covered. But those multifunction modem-routers rarely offer a stable experience, especially with a dozen WLAN bulbs.
Design
For basic infrastructure, I have a VDSL2 modem that can be put into bridge mode, and a PC Engines APU2 with a prosumer 5 GHz WLAN card and a cheap one for a fallback 2.4 GHz network. With my regular client devices, I want to end up with something like this:
Network-wise, in this configuration the router gets a public IP from the ISP, translates traffic from and to LAN IPs, and provides DHCP and a DNS cache for the LAN.
- Nftables is the current
iptables
replacement in Linux kernel for firewalling and routing. - DHCP is a stable simple-ish protocol, so ISC DHCP server is fine.
- For DNS cache, pihole is good for home networks. The router should still provide an internet DNS a fallback secondary.
- WLAN connectivity is done with
hostapd
, a bridge interface is created to link 2.4GHz and 5GHz WLAN interfaces with wired LAN interfaces. Nftables forwards traffic from wired WAN interface to the bridge interface and vice versa. - Home Assistant runs in a Docker container. My Zigbee coordinator is a cracked Lidl hub which interfaces its module over ethernet. HA's in-built ZHA is enough, but Zigbee2MQTT is an option if needed. I also have an extra hub for running both ZHA and Z2M at the same time.
- Central radio runs on Mopidy and Snapcast.
- File syncing is done with Syncthing, rather than e.g.
rsync
. Global discovery, hole punching, etc are turned off to ensure it is in a local-only mode. Syncthing on Android is easily hooked to sync files whenever connecting to home SSID.
To block WLAN IoT devices from the internet, an IP range will be set for MACs of those devices. This IP range is allowed only to communicate with other LAN IPs and multicast addresses. VLANs could also be used for this. They would allow coordinated filtering over multiple subnets, but is overkill for my purposes.
Implementation
Booting into Debian, the interfaces go as follows.
enp4s0
connects to the modem and the rest make the LAN.
Interface bridging and DHCP
We'll use 192.168.0.0/24
address space for our network. DHCP will handle other computers, but for the router we'll set 192.168.0.1
as static address. This IP is bound to the virtual interface bridging enp2s0
, enp3s0
, wlp1s0
and wlp5s0
. While hostapd
will link the wireless interfaces automatically to this bridging interface br0
, we'll bind the wired interfaces here. We'll add the following to /etc/network/interfaces
.
auto br0 enp4s0 iface enp2s0 inet manual iface enp3s0 inet manual iface enp4s0 inet dhcp iface br0 inet static bridge_ports enp2s0 enp3s0 address 192.168.0.1 broadcast 192.168.0.255 netmask 255.255.255.0 gateway 192.168.0.1
ISC DHCPd is configured in /etc/dhcp/dhcpd.conf
. I don't really care for in-LAN name resolution derived from computer names, so I've left domain-name
as default essentially. Fallback DNS is Cloudflare's 1.1.1.1
. The router and its DNS cache use ISP DNS received via enp4s0
.
option domain-name "lan"; option domain-name-servers 192.168.0.1, 1.1.1.1; default-lease-time 3600; max-lease-time 86400; ddns-update-style none; authoritative; subnet 192.168.0.0 netmask 255.255.255.0 { range 192.168.0.10 192.168.0.99; option routers 192.168.0.1; } host fed-imac-wired { hardware ethernet 3c:07:54:18:b7:51; fixed-address 192.168.0.7; } # .. etc # internet of shit - LAN-only host zigbee { hardware ethernet D4:A6:51:26:90:32; fixed-address 192.168.0.200; } # .. etc
IP routing
To translate network addresses, block non-essential ports, and block the IoT IP range, we'll set up /etc/nftables.conf
. First we'll block non-essential ports from internet access. The LAN is a trusted zone as I can't be bothered with configuring firewall whenever I try out some in-LAN service. By default we drop everything, but then accept everything from br0
and from select ports on the WAN interface (enp4s0
). I run SSH on port 404 to cut down on traffic. mosh
and Wireguard require opening some UDP ports.
#!/usr/sbin/nft -f flush ruleset table ip firewall { chain incoming { type filter hook input priority 0; policy drop; ct state established, related accept ct state invalid drop iifname "lo" accept iifname "br0" accept icmp type echo-request accept tcp dport { 80, 404, 443 } accept udp dport { 51820, 60000-61000 } accept } }
Then we'll set up stateful NAT using type nat hook
and masquerade
rules. Since it has a prerouting
chain, it can also handle blocking any traffic coming in br0
from .200-.250 range that has a destination address outside LAN.
table ip nat { chain prerouting { type nat hook prerouting priority 0; policy accept; iifname "br0" ip saddr { 192.168.0.200-192.168.0.250} \ ip daddr != { 192.168.0.0/24, 224.0.0.0/8, \ 239.255.255.0/24, 255.255.255.255 } drop } chain postrouting { type nat hook postrouting priority 100; policy accept; oifname "enp4s0" masquerade } }
DNS caching
I've previously run Pi-hole bare for straightforward setup. However, it doesn't use Debian's repositories making it too hacky (but modern and trendy) for my tastes. This time I'll use a Docker container, since I only need its DNS service on port 53 without web GUI or DHCP, and container abstracts the mess of an installation away, although pay attention on what interfaces ports
bind on. Docker will happily and silently poke holes in the firewall we set up above, if only e.g. 53:53/tcp
is used, exposing the resolver to the internet. Unfortunately in isn't easy to passthrough host's DNS records in Docker, so we'll have to hardcode my ISP's DNS servers, which luckily have remained static. We'll use the following docker-compose.yml
.
# More info at https://github.com/pi-hole/docker-pi-hole/ # and https://docs.pi-hole.net/ services: pihole: container_name: pihole image: pihole/pihole:latest ports: - "192.168.0.1:53:53/tcp" - "192.168.0.1:53:53/udp" environment: TZ: 'Europe/Helsinki' PIHOLE_DNS_: '1.2.3.4;2.3.4.5' DNSMASQ_LISTENING: 'all' FTLCONF_LOCAL_IPV4: '192.168.0.1' # Volumes store your data between container upgrades volumes: - './etc-pihole:/etc/pihole' - './etc-dnsmasq.d:/etc/dnsmasq.d' restart: unless-stopped
WLAN setup
hostapd
is a tool for running access points. To have both 2.4 GHz and 5 GHz networks, two different wireless cards and different SSIDs are needed. I have a laptop grade Qualcomm QCA6174
for the 2.4 GHz and an AP grade QCA9880
for 5 GHz. First, we configure the 2.4 GHz fallback network. I chose the channel by manually testing latency/bandwidth on a couple key channels. My building has a ton of 2.4G noise and I'm using it only for IoT and vintage devices. The key settings in my /etc/hostapd/wlp5s0.conf
are
interface=wlp5s0 driver=nl80211 ssid=tazca24 channel=11 hw_mode=g ht_capab=[DSSS_CCK-40][SHORT-GI-20][SHORT-GI-40][TX-STBC][RX-STBC1] vht_capab=[RX-STBC-1][TX-STBC][RXLDPC] auth_algs=1 ignore_broadcast_ssid=0 bridge=br0 ieee80211d=1 ieee80211h=1 ieee80211n=1
5 GHz is globally more regulated, and so cards have all kinds of doohickeys to determine correct regulations, and especially Intel ones are usually both extremely anal and confidently wrong about regulatory domains. We'll have to first find out where our QCA9880
thinks we are by running iw reg get
.
global country FI: DFS-ETSI phy#0 country US: DFS-FCC phy#1 country 99: DFS-UNSET
phy#0
is the 5G card, so we're apparently in the US. dmesg
says its EEPROM let ath
driver choose the domain. However it didn't set it to FI
which was set by iw
as global.
ath: EEPROM regdomain: 0x0 ath: EEPROM indicates default country code should be used ath: doing EEPROM country->regdmn map search ath: country maps to regdmn code: 0x3a ath: Country alpha2 being used: US ath: Regpair used: 0x3a
The US domain allows running the network on channel 42
, so we could go with that. However, according to Finnish regulations we are allowed to use 5150–5725 MHz band for a home WLAN. With 80 MHz bandwidth, that means channels 42
, 58
, 106
, 122
, and 138
, of which some will probably have less noise than 42
. Let's see how to make ath
understand the local regulations and my hardware.
Patching kernel
Following point 4.5 from the outdated Debian handbook on building custom kernel we can then patch the ath
driver and set DFS configuration flags.
$ grep -r "EEPROM indicates default" drivers drivers/net/wireless/ath/regd.c: printk(KERN_DEBUG "ath: EEPROM indicates default "
We can find reg->country_code
being set to CTRY_UNITED_STATES
by default. Let's check if Finland is CTRY_FINLAND
in the driver.
$ grep -r "CTRY_FINLAND" drivers drivers/net/wireless/ath/regd.h: CTRY_FINLAND = 246
Changing the constant, a patch
goes as follows.
--- a/drivers/net/wireless/ath/regd.c 2024-05-30 10:49:53.000000000 +0300 +++ b/drivers/net/wireless/ath/regd.c 2024-06-29 13:35:08.860685835 +0300 @@ -704,7 +704,7 @@ regdmn == CTRY_DEFAULT) { printk(KERN_DEBUG "ath: EEPROM indicates default " "country code should be used\n"); - reg->country_code = CTRY_UNITED_STATES; + reg->country_code = CTRY_FINLAND; } if (reg->country_code == CTRY_DEFAULT) {
I'll just apply the patch straight without using quilt
.
patch -p1 < ../ath_default_country_fi.patch
To allow using DFS bands (58–144), we need to tell the driver my QCA9880 module is DFS certified. To do so, we add following flags to .config
I've copied /boot/config-6.8.12-amd64
to.
CONFIG_CFG80211_CERTIFICATION_ONUS=y CONFIG_ATH10K_DFS_CERTIFIED=y
make clean && make -j12 bindeb-pkg
Setting up 5 GHz
After booting into the kernel, we can see our allowed bands and transmission powers. radar detection
means using DFS. Strangely, as I understand it, no IR
(no initiating radiation) should deny forming an access point. I assume iw
doesn't get authoritative information.
* 5180.0 MHz [36] (23.0 dBm) * 5200.0 MHz [40] (23.0 dBm) * 5220.0 MHz [44] (23.0 dBm) * 5240.0 MHz [48] (23.0 dBm) * 5260.0 MHz [52] (20.0 dBm) (no IR, radar detection) * 5280.0 MHz [56] (20.0 dBm) (no IR, radar detection) * 5300.0 MHz [60] (20.0 dBm) (no IR, radar detection) * 5320.0 MHz [64] (20.0 dBm) (no IR, radar detection) * 5500.0 MHz [100] (26.0 dBm) (no IR, radar detection) * 5520.0 MHz [104] (26.0 dBm) (no IR, radar detection) * 5540.0 MHz [108] (26.0 dBm) (no IR, radar detection) * 5560.0 MHz [112] (26.0 dBm) (no IR, radar detection) * 5580.0 MHz [116] (26.0 dBm) (no IR, radar detection) * 5600.0 MHz [120] (26.0 dBm) (no IR, radar detection) * 5620.0 MHz [124] (26.0 dBm) (no IR, radar detection) * 5640.0 MHz [128] (26.0 dBm) (no IR, radar detection) * 5660.0 MHz [132] (26.0 dBm) (no IR, radar detection) * 5680.0 MHz [136] (26.0 dBm) (no IR, radar detection) * 5700.0 MHz [140] (26.0 dBm) (no IR, radar detection) * 5720.0 MHz [144] (13.0 dBm) (radar detection) * 5745.0 MHz [149] (13.0 dBm) * 5765.0 MHz [153] (13.0 dBm) * 5785.0 MHz [157] (13.0 dBm) * 5805.0 MHz [161] (13.0 dBm) * 5825.0 MHz [165] (13.0 dBm) * 5845.0 MHz [169] (13.0 dBm) * 5865.0 MHz [173] (13.0 dBm)
Bands 106
, 122
, and 138
are the most lucrative options with their higher 26 dBm limit. With iperf3
the performance seems to fluctuate between 200–500 Mbps on all bands over an hour, but 122
seems to average closest to 500 Mbps with 106
close second. 42
has about 20% less bandwidth and 138
50% less or so. 58
is close contender, but signal strength seems to be noticeably impacted on my phone.
Theoretical maximum for this network is 1300 Mbps, so I'm happy with the result. Something not related to signal quality is bottlenecking it at 500~520 Mbps, all bands occasionally reach that, but have intermittent noise in them. So, I ended up with following key settings in /etc/hostapd/wlp1s0.conf
.
interface=wlp1s0 driver=nl80211 ssid=tazca channel=116 hw_mode=a auth_algs=1 ignore_broadcast_ssid=0 bridge=br0 # DFS ieee80211d=1 ieee80211h=1 # 802.11n ieee80211n=1 ht_capab=[DSSS_CCK-40][HT40+][SHORT-GI-20][SHORT-GI-40][TX-STBC] [RX-STBC1][DSSS_CCK-40][LDPC][MAX-AMSDU-7935] # 802.11ac ieee80211ac=1 vht_capab=[SHORT-GI-80][RX-ANTENNA-PATTERN][TX-ANTENNA-PATTERN] [RX-STBC-1][TX-STBC-2BY1][RXLDPC][MAX-A-MPDU-LEN-EXP7][MAX-MPDU-11454] vht_oper_chwidth=1 vht_oper_centr_freq_seg0_idx=122
The networks are then started using hostapd@wlp1s0.service
and hostapd@wlp5s0.service
(on Systemd). With the network now humming along, we can get to actual user services.
IoT, radio, file sync, other services
With any IP protocol IoT devices/hubs blocked from internet, Home Assistant can be set up. docker-compose.yml
:
services: homeassistant: container_name: homeassistant image: ghcr.io/home-assistant/home-assistant:stable privileged: true network_mode: host devices: - /dev/ttyzbbridge1:/dev/ttyzbbridge1 environment: TZ: 'Europe/Helsinki' volumes: - './ha_volume:/config' restart: unless-stopped
The central radio runs Mopidy from a python virtual environment. Its development has been slow and it's buggy, but Mopidy allows streaming from Youtube, Bandcamp and Tidal from a web UI like Mopidy-Iris
. When it works. Mopidy-Bandcamp
, Mopidy-youtube
and yt-dlp
are better as nightly versions straight from GitHub.
$ pip install --upgrade mopidy Mopidy-MPD Mopidy-SoundCloud Mopidy-Iris Mopidy-Local Mopidy-Tidal pygobject $ pip install --force-reinstall https://github.com/impliedchaos/mopidy-bandcamp/archive/refs/heads/master.zip $ pip install --force-reinstall https://github.com/natumbri/mopidy-youtube/archive/refs/heads/develop.zip $ pip install --force-reinstall https://github.com/yt-dlp/yt-dlp/archive/master.tar.gz
Snapcast server is run as a system Systemd service, but I also want to have Bluetooth speakers automatically hook up and play when needed. Enabling linger
allows user services like wireplumber
and snapclient
to start without logging in.
# loginctl enable-linger username
Wireplumber also requires disabling seat monitoring in e.g. ~/.config/wireplumber/wireplumber.conf.d/80-disable-logind.conf
.
wireplumber.profiles = { main = { monitor.bluez.seat-monitoring = disabled } }
In Syncthing, to disable global discovery, put false
in <globalAnnounceEnabled>
in ~/.local/state/syncthing/config.xml
.