The same sysctl file, three different machines

I run the same core network tuning across the NAS (Fedora Server), the desktop (NixOS), and a laptop or two … not because every machine has the same workload, but because the underlying kernel-level wins are the same regardless of what’s actually running on top. The differences show up in what else gets layered on, not in the base config.

BBR + CAKE, together

The single biggest win of the lot. tcp_congestion_control defaults to cubic on most distros, which is fine but not great, especially over Tailscale/WireGuard tunnels where the effective path characteristics don’t always match what cubic assumes. bbr measures actual bandwidth and round-trip time instead of just reacting to packet loss, and pairs specifically well with cake as the queuing discipline … cake handles bufferbloat and fair queuing far better than the kernel’s default fq or pfifo_fast.

net.core.default_qdisc              = cake
net.ipv4.tcp_congestion_control     = bbr

Applied identically everywhere. This one isn’t workload-specific … every machine benefits from better congestion control and less bufferbloat, no exceptions.

CAKE needs an actual number, not just a name

Here’s the bit I missed at first. Setting default_qdisc = cake in sysctl only makes CAKE the default … it doesn’t actually configure any bandwidth shaping on its own. Without a real bandwidth figure, CAKE is running mostly blind on the exact thing it’s supposed to be best at, which is fighting bufferbloat.

The actual fix is setting it explicitly on the interface, with a number close to real-world throughput, not the theoretical link speed:

sudo tc qdisc replace dev enp2s0 root cake bandwidth 900mbit

Gigabit link, but realistically closer to 940mbit under ideal conditions … 900mbit gives CAKE a number to actually manage queue depth against instead of guessing.

tc commands don’t survive a reboot by themselves, so it needs a small service to reapply it at boot:

sudo tee /etc/systemd/system/cake-shaping.service << 'EOF'
[Unit]
Description=Apply my CAKE bandwidth shaping
After=network-online.target
Wants=network-online.target

[Service]
Type=oneshot
ExecStart=/usr/sbin/tc qdisc replace dev enp2s0 root cake bandwidth 900mbit
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target
EOF
sudo systemctl daemon-reload
sudo systemctl enable --now cake-shaping.service

Confirm it’s actually doing something, not just assigned:

tc -s qdisc show dev enp2s0

Real stats show up there … drops, marks, backlog — if there’s ever congestion to actually manage.

Network buffers, sized for the actual pipe

Default TCP buffer sizes are conservative, tuned for a much wider range of hardware than what any single machine actually has. Since I’m mostly moving real file traffic … Samba, NFS, SFTP … over gigabit LAN and Tailscale, giving the kernel room to actually use the available bandwidth matters:

net.core.rmem_max                   = 134217728
net.core.wmem_max                   = 134217728
net.core.rmem_default               = 262144
net.core.wmem_default               = 262144
net.ipv4.tcp_rmem                   = 4096 87380 134217728
net.ipv4.tcp_wmem                   = 4096 65536 134217728
net.core.netdev_max_backlog         = 5000
net.core.somaxconn                  = 1024

Same values across all three machines. The ceiling doesn’t hurt anything on a weaker box … it’s a maximum, not a floor, the kernel still scales buffers dynamically underneath that.

MTU black-hole avoidance

This one’s specifically relevant if you’re running anything over Tailscale or another VPN/tunnel … some network paths silently drop the ICMP messages that normal path MTU discovery depends on, which causes large transfers to hang intermittently for no obvious reason. It’s off by default in the kernel, and turning it on doesn’t cost anything unless a black hole actually gets detected, at which point it kicks in automatically:

net.ipv4.tcp_mtu_probing             = 1

Cheap insurance. I added this specifically after digging into some inconsistent transfer behaviour over Tailscale and realising it was exactly the kind of thing this setting exists to catch.

Where the NAS-specific tuning splits off

Everything above is universal. The NAS gets a few extra lines the desktop and laptop don’t need, because it’s actually serving NFS traffic instead of just consuming it:

sunrpc.tcp_slot_table_entries       = 128
sunrpc.udp_slot_table_entries       = 128

These tune NFS client-side performance specifically … relevant on the NAS because it’s mounting shares from another NAS behind it, and on any desktop that mounts NFS shares from the NAS itself. Doesn’t do anything on a machine that never touches NFS.

Memory tuning … same logic, different numbers per machine

vm.swappiness and friends aren’t really “networking” but they live in the same sysctl file, and the values genuinely differ by machine, on purpose:

vm.swappiness                       = 10
vm.dirty_ratio                      = 15
vm.dirty_background_ratio           = 5
vm.vfs_cache_pressure               = 50

On the file server (T620, 12GB RAM, no interactive session to protect), low swappiness just means the kernel doesn’t reach for swap until it genuinely has to, which is exactly what you want. The QNAP behind it doesn’t get any of this tuning at all … it’s just the storage backend running its own QTS firmware, nothing here applies to it. Lower-RAM machines (an old laptop with 8GB, say) get more conservative settings across the board … this isn’t a copy-paste-everywhere file, the RAM ceiling of each box changes what’s actually safe to set aggressively.

The one thing I don’t copy-paste blindly

Socket options at the Samba level are a separate, related trap worth mentioning here since it looks similar but isn’t sysctl at all. I used to hardcode SO_RCVBUF/SO_SNDBUF values in smb.conf, thinking I was helping. Turns out fixed buffer sizes there actively override the kernel’s own auto-tuning … the same tcp_rmem/tcp_wmem scaling described above … and can throttle a fast, low-latency connection that would’ve done better left alone:

socket options          = TCP_NODELAY

Just that. No manual buffer sizes. Let the sysctl tuning above do the actual work, and Samba just gets out of the way.

Applying it

sudo tee /etc/sysctl.d/99-network-tuning.conf << 'EOF'
net.core.default_qdisc              = cake
net.ipv4.tcp_congestion_control     = bbr
net.core.rmem_max                   = 134217728
net.core.wmem_max                   = 134217728
net.core.rmem_default               = 262144
net.core.wmem_default               = 262144
net.ipv4.tcp_rmem                   = 4096 87380 134217728
net.ipv4.tcp_wmem                   = 4096 65536 134217728
net.core.netdev_max_backlog         = 5000
net.core.somaxconn                  = 1024
net.ipv4.tcp_mtu_probing            = 1
EOF
sudo sysctl --system

Verify it actually took, since sysctl silently ignoring a typo is a real way to think something’s tuned when it isn’t:

sysctl net.ipv4.tcp_congestion_control
sysctl net.core.default_qdisc
sysctl net.ipv4.tcp_mtu_probing

Same base file, three machines, small deliberate differences where the workload actually calls for it … that’s the whole approach.