Tatu Projects Journal
Setting up a home LAN

Last update: 2024-06-30

Motivation

Cloud services are ubiquitous these days and even sensitive material can be passed through those with appropriate encryption. However, in my home, I want stable solutions that are not susceptible to bankcrupties, buyouts, or connectivity/remote server issues. I want to have a local-first and customizable network that allows me to:

  • Have stable 2.4/5GHz WLAN and 1Gbit LAN connectivity, with customizable routing/firewall tables, DHCP and a local DNS cache
  • Sync files between devices at LAN speeds, primarily Org and offline music, and offloading photos from my phone
  • Handle Internet of Things devices locally (via Home Assistant); allow isolating them from internet
  • Run a central radio

With IPv4 still dominating internet traffic, each household usually has a NAT already, so basic connectivity is often covered. But those multifunction modem-routers rarely offer a stable experience, especially with a dozen WLAN bulbs.

Design

For basic infrastructure, I have a VDSL2 modem that can be put into bridge mode, and a PC Engines APU2 with a prosumer 5 GHz WLAN card and a cheap one for a fallback 2.4 GHz network. With my regular client devices, I want to end up with something like this:

WorkstationPCMobileRouterModemInternetLampPowersocketThermometertazca.comFedora+Win10Fedora+MacOSDebianGrapheneOSZyXEL proprietaryDebianZigbee

Network-wise, in this configuration the router gets a public IP from the ISP, translates traffic from and to LAN IPs, and provides DHCP and a DNS cache for the LAN.

  • Nftables is the current iptables replacement in Linux kernel for firewalling and routing.
  • DHCP is a stable simple-ish protocol, so ISC DHCP server is fine.
  • For DNS cache, pihole is good for home networks. The router should still provide an internet DNS a fallback secondary.
  • WLAN connectivity is done with hostapd, a bridge interface is created to link 2.4GHz and 5GHz WLAN interfaces with wired LAN interfaces. Nftables forwards traffic from wired WAN interface to the bridge interface and vice versa.
  • Home Assistant runs in a Docker container. My Zigbee coordinator is a cracked Lidl hub which interfaces its module over ethernet. HA's in-built ZHA is enough, but Zigbee2MQTT is an option if needed. I also have an extra hub for running both ZHA and Z2M at the same time.
  • Central radio runs on Mopidy and Snapcast.
  • File syncing is done with Syncthing, rather than e.g. rsync. Global discovery, hole punching, etc are turned off to ensure it is in a local-only mode. Syncthing on Android is easily hooked to sync files whenever connecting to home SSID.

To block WLAN IoT devices from the internet, an IP range will be set for MACs of those devices. This IP range is allowed only to communicate with other LAN IPs and multicast addresses. VLANs could also be used for this. They would allow coordinated filtering over multiple subnets, but is overkill for my purposes.

Implementation

Booting into Debian, the interfaces go as follows.

2024-06-network-interfaces.webp

enp4s0 connects to the modem and the rest make the LAN.

Interface bridging and DHCP

We'll use 192.168.0.0/24 address space for our network. DHCP will handle other computers, but for the router we'll set 192.168.0.1 as static address. This IP is bound to the virtual interface bridging enp2s0, enp3s0, wlp1s0 and wlp5s0. While hostapd will link the wireless interfaces automatically to this bridging interface br0, we'll bind the wired interfaces here. We'll add the following to /etc/network/interfaces.

auto br0 enp4s0
iface enp2s0 inet manual
iface enp3s0 inet manual
iface enp4s0 inet dhcp

iface br0 inet static
    bridge_ports enp2s0 enp3s0
    address 192.168.0.1
    broadcast 192.168.0.255
    netmask 255.255.255.0
    gateway 192.168.0.1

ISC DHCPd is configured in /etc/dhcp/dhcpd.conf. I don't really care for in-LAN name resolution derived from computer names, so I've left domain-name as default essentially. Fallback DNS is Cloudflare's 1.1.1.1. The router and its DNS cache use ISP DNS received via enp4s0.

option domain-name "lan";
option domain-name-servers 192.168.0.1, 1.1.1.1;

default-lease-time 3600;
max-lease-time 86400;

ddns-update-style none;
authoritative;

subnet 192.168.0.0 netmask 255.255.255.0 {
  range 192.168.0.10 192.168.0.99;
  option routers 192.168.0.1;
}

host fed-imac-wired {
  hardware ethernet 3c:07:54:18:b7:51;
  fixed-address 192.168.0.7;
} # .. etc

# internet of shit - LAN-only
host zigbee {
  hardware ethernet D4:A6:51:26:90:32;
  fixed-address 192.168.0.200;
} # .. etc

IP routing

To translate network addresses, block non-essential ports, and block the IoT IP range, we'll set up /etc/nftables.conf. First we'll block non-essential ports from internet access. The LAN is a trusted zone as I can't be bothered with configuring firewall whenever I try out some in-LAN service. By default we drop everything, but then accept everything from br0 and from select ports on the WAN interface (enp4s0). I run SSH on port 404 to cut down on traffic. mosh and Wireguard require opening some UDP ports.

#!/usr/sbin/nft -f

flush ruleset

table ip firewall {
        chain incoming {
                type filter hook input priority 0; policy drop;
                ct state established, related accept
                ct state invalid drop
                iifname "lo" accept
                iifname "br0" accept
                icmp type echo-request accept
                tcp dport { 80, 404, 443 } accept
                udp dport { 51820, 60000-61000 } accept
        }
}

Then we'll set up stateful NAT using type nat hook and masquerade rules. Since it has a prerouting chain, it can also handle blocking any traffic coming in br0 from .200-.250 range that has a destination address outside LAN.

table ip nat {
        chain prerouting {
                type nat hook prerouting priority 0; policy accept;
                iifname "br0" ip saddr { 192.168.0.200-192.168.0.250} \
                              ip daddr != { 192.168.0.0/24, 224.0.0.0/8, \
                                            239.255.255.0/24, 255.255.255.255 } drop
        }

        chain postrouting {
                type nat hook postrouting priority 100; policy accept;
                oifname "enp4s0" masquerade
        }
}

DNS caching

I've previously run Pi-hole bare for straightforward setup. However, it doesn't use Debian's repositories making it too hacky (but modern and trendy) for my tastes. This time I'll use a Docker container, since I only need its DNS service on port 53 without web GUI or DHCP, and container abstracts the mess of an installation away, although pay attention on what interfaces ports bind on. Docker will happily and silently poke holes in the firewall we set up above, if only e.g. 53:53/tcp is used, exposing the resolver to the internet. Unfortunately in isn't easy to passthrough host's DNS records in Docker, so we'll have to hardcode my ISP's DNS servers, which luckily have remained static. We'll use the following docker-compose.yml.

# More info at https://github.com/pi-hole/docker-pi-hole/
# and https://docs.pi-hole.net/
services:
  pihole:
    container_name: pihole
    image: pihole/pihole:latest
    ports:
      - "192.168.0.1:53:53/tcp"
      - "192.168.0.1:53:53/udp"
    environment:
      TZ: 'Europe/Helsinki'
      PIHOLE_DNS_: '1.2.3.4;2.3.4.5'
      DNSMASQ_LISTENING: 'all'
      FTLCONF_LOCAL_IPV4: '192.168.0.1'
    # Volumes store your data between container upgrades
    volumes:
      - './etc-pihole:/etc/pihole'
      - './etc-dnsmasq.d:/etc/dnsmasq.d'
    restart: unless-stopped

WLAN setup

hostapd is a tool for running access points. To have both 2.4 GHz and 5 GHz networks, two different wireless cards and different SSIDs are needed. I have a laptop grade Qualcomm QCA6174 for the 2.4 GHz and an AP grade QCA9880 for 5 GHz. First, we configure the 2.4 GHz fallback network. I chose the channel by manually testing latency/bandwidth on a couple key channels. My building has a ton of 2.4G noise and I'm using it only for IoT and vintage devices. The key settings in my /etc/hostapd/wlp5s0.conf are

interface=wlp5s0
driver=nl80211
ssid=tazca24
channel=11
hw_mode=g
ht_capab=[DSSS_CCK-40][SHORT-GI-20][SHORT-GI-40][TX-STBC][RX-STBC1]
vht_capab=[RX-STBC-1][TX-STBC][RXLDPC]

auth_algs=1
ignore_broadcast_ssid=0
bridge=br0
ieee80211d=1
ieee80211h=1
ieee80211n=1

5 GHz is globally more regulated, and so cards have all kinds of doohickeys to determine correct regulations, and especially Intel ones are usually both extremely anal and confidently wrong about regulatory domains. We'll have to first find out where our QCA9880 thinks we are by running iw reg get.

global
country FI: DFS-ETSI

phy#0
country US: DFS-FCC

phy#1
country 99: DFS-UNSET

phy#0 is the 5G card, so we're apparently in the US. dmesg says its EEPROM let ath driver choose the domain. However it didn't set it to FI which was set by iw as global.

ath: EEPROM regdomain: 0x0
ath: EEPROM indicates default country code should be used
ath: doing EEPROM country->regdmn map search
ath: country maps to regdmn code: 0x3a
ath: Country alpha2 being used: US
ath: Regpair used: 0x3a

The US domain allows running the network on channel 42, so we could go with that. However, according to Finnish regulations we are allowed to use 5150–5725 MHz band for a home WLAN. With 80 MHz bandwidth, that means channels 42, 58, 106, 122, and 138, of which some will probably have less noise than 42. Let's see how to make ath understand the local regulations and my hardware.

Patching kernel

Following point 4.5 from the outdated Debian handbook on building custom kernel we can then patch the ath driver and set DFS configuration flags.

$ grep -r "EEPROM indicates default" drivers
drivers/net/wireless/ath/regd.c:
    printk(KERN_DEBUG "ath: EEPROM indicates default "

We can find reg->country_code being set to CTRY_UNITED_STATES by default. Let's check if Finland is CTRY_FINLAND in the driver.

$ grep -r "CTRY_FINLAND" drivers
drivers/net/wireless/ath/regd.h:
    CTRY_FINLAND = 246

Changing the constant, a patch goes as follows.

--- a/drivers/net/wireless/ath/regd.c   2024-05-30 10:49:53.000000000 +0300
+++ b/drivers/net/wireless/ath/regd.c   2024-06-29 13:35:08.860685835 +0300
@@ -704,7 +704,7 @@
        regdmn == CTRY_DEFAULT) {
        printk(KERN_DEBUG "ath: EEPROM indicates default "
               "country code should be used\n");
-       reg->country_code = CTRY_UNITED_STATES;
+       reg->country_code = CTRY_FINLAND;
    }

    if (reg->country_code == CTRY_DEFAULT) {

I'll just apply the patch straight without using quilt.

patch -p1 < ../ath_default_country_fi.patch

To allow using DFS bands (58–144), we need to tell the driver my QCA9880 module is DFS certified. To do so, we add following flags to .config I've copied /boot/config-6.8.12-amd64 to.

CONFIG_CFG80211_CERTIFICATION_ONUS=y
CONFIG_ATH10K_DFS_CERTIFIED=y
make clean && make -j12 bindeb-pkg

Setting up 5 GHz

After booting into the kernel, we can see our allowed bands and transmission powers. radar detection means using DFS. Strangely, as I understand it, no IR (no initiating radiation) should deny forming an access point. I assume iw doesn't get authoritative information.

* 5180.0 MHz [36] (23.0 dBm)
* 5200.0 MHz [40] (23.0 dBm)
* 5220.0 MHz [44] (23.0 dBm)
* 5240.0 MHz [48] (23.0 dBm)
* 5260.0 MHz [52] (20.0 dBm) (no IR, radar detection)
* 5280.0 MHz [56] (20.0 dBm) (no IR, radar detection)
* 5300.0 MHz [60] (20.0 dBm) (no IR, radar detection)
* 5320.0 MHz [64] (20.0 dBm) (no IR, radar detection)
* 5500.0 MHz [100] (26.0 dBm) (no IR, radar detection)
* 5520.0 MHz [104] (26.0 dBm) (no IR, radar detection)
* 5540.0 MHz [108] (26.0 dBm) (no IR, radar detection)
* 5560.0 MHz [112] (26.0 dBm) (no IR, radar detection)
* 5580.0 MHz [116] (26.0 dBm) (no IR, radar detection)
* 5600.0 MHz [120] (26.0 dBm) (no IR, radar detection)
* 5620.0 MHz [124] (26.0 dBm) (no IR, radar detection)
* 5640.0 MHz [128] (26.0 dBm) (no IR, radar detection)
* 5660.0 MHz [132] (26.0 dBm) (no IR, radar detection)
* 5680.0 MHz [136] (26.0 dBm) (no IR, radar detection)
* 5700.0 MHz [140] (26.0 dBm) (no IR, radar detection)
* 5720.0 MHz [144] (13.0 dBm) (radar detection)
* 5745.0 MHz [149] (13.0 dBm)
* 5765.0 MHz [153] (13.0 dBm)
* 5785.0 MHz [157] (13.0 dBm)
* 5805.0 MHz [161] (13.0 dBm)
* 5825.0 MHz [165] (13.0 dBm)
* 5845.0 MHz [169] (13.0 dBm)
* 5865.0 MHz [173] (13.0 dBm)

Bands 106, 122, and 138 are the most lucrative options with their higher 26 dBm limit. With iperf3 the performance seems to fluctuate between 200–500 Mbps on all bands over an hour, but 122 seems to average closest to 500 Mbps with 106 close second. 42 has about 20% less bandwidth and 138 50% less or so. 58 is close contender, but signal strength seems to be noticeably impacted on my phone.

Theoretical maximum for this network is 1300 Mbps, so I'm happy with the result. Something not related to signal quality is bottlenecking it at 500~520 Mbps, all bands occasionally reach that, but have intermittent noise in them. So, I ended up with following key settings in /etc/hostapd/wlp1s0.conf.

interface=wlp1s0
driver=nl80211
ssid=tazca
channel=116
hw_mode=a
auth_algs=1
ignore_broadcast_ssid=0
bridge=br0

# DFS
ieee80211d=1
ieee80211h=1

# 802.11n
ieee80211n=1
ht_capab=[DSSS_CCK-40][HT40+][SHORT-GI-20][SHORT-GI-40][TX-STBC]
[RX-STBC1][DSSS_CCK-40][LDPC][MAX-AMSDU-7935]

# 802.11ac
ieee80211ac=1
vht_capab=[SHORT-GI-80][RX-ANTENNA-PATTERN][TX-ANTENNA-PATTERN]
[RX-STBC-1][TX-STBC-2BY1][RXLDPC][MAX-A-MPDU-LEN-EXP7][MAX-MPDU-11454]
vht_oper_chwidth=1
vht_oper_centr_freq_seg0_idx=122

The networks are then started using hostapd@wlp1s0.service and hostapd@wlp5s0.service (on Systemd). With the network now humming along, we can get to actual user services.

IoT, radio, file sync, other services

With any IP protocol IoT devices/hubs blocked from internet, Home Assistant can be set up. docker-compose.yml:

services:
  homeassistant:
    container_name: homeassistant
    image: ghcr.io/home-assistant/home-assistant:stable
    privileged: true
    network_mode: host
    devices:
      - /dev/ttyzbbridge1:/dev/ttyzbbridge1
    environment:
      TZ: 'Europe/Helsinki'
    volumes:
      - './ha_volume:/config'
    restart: unless-stopped

The central radio runs Mopidy from a python virtual environment. Its development has been slow and it's buggy, but Mopidy allows streaming from Youtube, Bandcamp and Tidal from a web UI like Mopidy-Iris. When it works. Mopidy-Bandcamp, Mopidy-youtube and yt-dlp are better as nightly versions straight from GitHub.

$ pip install --upgrade mopidy Mopidy-MPD Mopidy-SoundCloud Mopidy-Iris Mopidy-Local Mopidy-Tidal pygobject
$ pip install --force-reinstall https://github.com/impliedchaos/mopidy-bandcamp/archive/refs/heads/master.zip
$ pip install --force-reinstall https://github.com/natumbri/mopidy-youtube/archive/refs/heads/develop.zip
$ pip install --force-reinstall https://github.com/yt-dlp/yt-dlp/archive/master.tar.gz

Snapcast server is run as a system Systemd service, but I also want to have Bluetooth speakers automatically hook up and play when needed. Enabling linger allows user services like wireplumber and snapclient to start without logging in.

# loginctl enable-linger username

Wireplumber also requires disabling seat monitoring in e.g. ~/.config/wireplumber/wireplumber.conf.d/80-disable-logind.conf.

wireplumber.profiles = {
  main = {
    monitor.bluez.seat-monitoring = disabled
  }
}

In Syncthing, to disable global discovery, put false in <globalAnnounceEnabled> in ~/.local/state/syncthing/config.xml.