Skip to main content

nyxd networking

This document describes how the daemon picks a network implementation, how that maps to code, and how to run with native (default) or CNI plugins.

Who chooses the driver?​

Only nyxd (the daemon) selects the network backend via CLI flags. The nyx client does not set -net-driver; it talks to the control socket. After you change networking flags, rebuild and restart nyxd so the new binary and options are in effect.

Host firewalls (UFW / FORWARD DROP)​

Native networking requires net.bridge.bridge-nf-call-iptables=1, so bridged container traffic still hits the host filter FORWARD hook. Embedded DNS on the bridge gateway hits INPUT (UDP/TCP 53).

If the host uses UFW (or any firewall) with DEFAULT_FORWARD_POLICY=DROP / restrictive INPUT, lookups like storage.nyxd.local from a container time out (read udp …→10.88.0.1:53: i/o timeout) and east-west service traffic fails — even when dig @10.88.0.1 works from the host itself.

nyxd installs ACCEPT rules (comment prefix nyxd-fw-) at the head of iptables FORWARD/INPUT for each bridge (nyxbr0 and compose custom bridges), ahead of UFW’s jumps:

  • FORWARD hairpin: -i BRIDGE -o BRIDGE
  • FORWARD egress: -i BRIDGE
  • FORWARD return: -o BRIDGE RELATED,ESTABLISHED
  • INPUT DNS: -i BRIDGE UDP/TCP dport 53

Rules are re-asserted on every bridge ensure (container start / EnsureNetwork), so a ufw reload that wipes them is repaired on the next setup. Verify with:

sudo iptables -L FORWARD -n -v --line-numbers | head
sudo iptables -L INPUT -n -v --line-numbers | head

Workaround if iptables is unavailable: allow the bridge in UFW manually, or run compose services with network_mode: host (no bridge DNS).

DNS (-dns)​

nyxd can run a small embedded DNS server on the bridge gateway (UDP port 53, default NYXD_GATEWAY_IP / 10.88.0.1) that answers A records for container hostnames (<hostname> and <hostname>.nyxd.local). Containers receive a bind-mounted /etc/resolv.conf pointing at that gateway when registration applies.

-dns valueBehaviour
auto (default)Native (-net-driver=native): embedded DNS for every supervised container. CNI: embedded DNS only for Compose stacks where each service is attached solely to networks declared with internal: true (see compose networks:).
embeddedAlways use embedded DNS (including all services on CNI).
cniDo not run embedded DNS; use CNI-side DNS (for example the dnsname plugin in your conflist).
offNo embedded DNS and no /etc/resolv.conf override from nyxd.

Set NYXD_GATEWAY_IP (and NYXD_CONTAINER_SUBNET) to match the bridge your CNI plugin uses if it differs from the native defaults.

Compose (native): a service may set dns: (IPv4 nameservers) or mount /etc/resolv.conf; either replaces the gateway resolv file. A networks: entry with both bridge: and subnet: gets its own host bridge and an embedded DNS zone on that bridge's gateway; services that omit networks stay on nyxbr0. See native-network.md.

AppArmor: apparmor_profile is applied only when the host has AppArmor and the named profile is loaded. On SELinux-only or Alpine hosts the field is ignored (warning logged) so the same compose file does not fail crun. macOS/Windows hosts have no AppArmor (the CLI still runs; the runtime is Linux inside nyx machine) — see Platform matrix.

Control API: GET /v1/network returns the active driver, default bridge, gateway, and subnet.

Default: native (no /opt/cni/bin)​

By default, nyxd runs with:

sudo ./nyxd --log-level info
# equivalent explicit flag:
# sudo ./nyxd --net-driver=native --log-level info

On startup you should see a JSON log line similar to:

{"msg":"network backend","driver":"native",...}

Then run a container:

sudo ./nyx pull nginx:alpine
sudo ./nyx run nginx:alpine

Foreground vs detach: by default, nyx run prints the JSON response, then waits; Ctrl+C (or SIGTERM) calls POST /v1/containers/{id}/stop so the workload is torn down (similar to docker run without -d). Use nyx run -d / --detach to exit immediately after start (old behavior). You can also run nyx stop <id> from another terminal.

Native networking is implemented under internal/network/native/ (bridge, veth, IPAM, nftables-based port maps). It does not require the standard CNI plugin bundle under /opt/cni/bin.

Host network (-net-driver=host)​

Containers share the host network namespace: no bridge, no NAT, no netns, no extra binaries. It is the default (and only) driver for rootless nyxd, and can be selected explicitly when rootful.

  • -p/--publish is ignored with a warning; services bind host ports directly.
  • The host /etc/resolv.conf is bind-mounted read-only; embedded DNS (-dns) is off.
  • Containers see each other on 127.0.0.1.

Optional: CNI plugins (-net-driver=cni)​

To use the exec-based path that shells out to standard CNI plugins (bridge, host-local, etc.), start the daemon with:

sudo ./nyxd -net-driver=cni \
  -cni-bin-dir=/opt/cni/bin \
  -cni-conf-dir=/etc/cni/net.d \
  -network=nyx \
  --log-level info

Install plugins first, for example:

make install-cni
# or install containernetworking/plugins to /opt/cni/bin by your own process

You should then see:

{"msg":"network backend","driver":"cni",...}

Code layout: network.Backend and the supervisor​

network.Backend​

The interface is defined in internal/network/backend.go:

  • EnsureNetwork() — host-wide setup (bridge / CNI conflist, etc.)
  • Setup(ctx, containerID, netNSPath, ports, opts *SetupOptions) — join the container netns, return allocated IP (or equivalent). opts.Internal (compose networks.*.internal: true) is enforced on the native driver by nftables forward drops for IPv4 traffic leaving the bridge CIDR; CNI ignores opts for now (configure plugins separately).
  • Teardown(ctx, containerID, netNSPath) — tear down that container’s networking

Implementations:

ImplementationPackage / typeNotes
Nativeinternal/network/native — *native.ManagerIn-process; no CNI binaries on disk
CNI execinternal/network — *network.ManagerRuns plugins from -cni-bin-dir

Both satisfy network.Backend (see compile-time assertions in native/manager.go and network/cni.go).

Supervisor​

internal/supervisor holds a network.Backend, not a concrete *network.Manager. supervisor.New(...) also receives the selected DNS backend and -dns / -net-driver policy for compose stack registration; see DNS (-dns) above.

network.PortMapping​

supervisor.ContainerSpec uses []network.PortMapping. The native stack imports and uses network.PortMapping as well, so port maps from the API/supervisor align with native setup without duplicate structs.

systemd​

The packaged service (packaging/linux/nyxd.service) runs the default native driver. To use CNI, set NYXD_OPTS="-net-driver=cni" in /etc/default/nyxd (and make sure /opt/cni/bin exists), then restart the service.