Build Your Own FreeBSD Server

About Me

Brief Introduction -- Nils Imhoff

• Works as a DevOps-Engineer
• @work: mostly Ansible, Terraform, K8s, Linux
• @freetime: mostly BSD
• Reachable via nils@d4d1.de

My Repository

For the tutorial, I have prepared a repository:

git clone ssh://anonymous@got.d4d1.de:2225/bsdcan-tutorial-2026.git

and my slides

as pdf

Target picture

Console-based private cloud and NAS under your control:

• zroot · work · data
• Bastille + VNET on shared bridges
• PF, file sharing, snapshots, updates

Schedule

SSO, Tailscale, OPNsense HAProxy + ACME, Cloud Services, podman

Lab

ssh tutorial@vmNN.d4d1.de · nda0–nda4 · handouts

Part 1 — Basics & ZFS

separate system / work / data

Pools

nda0      → zroot
nda1+2    → work  (mirror)  → bastille, bhyve
nda3+4    → data  (mirror)  → home, media, backups

At home: data is often raidz2 on many disks

work datasets

Basics

• pkg update && pkg upgrade
• SSH: no root, no passwords (production)
• bectl create pre-tutorial

Part 2 — Networking & PF

SMART · Monit · VNET · bridges · firewall rules

S.M.A.R.T. + Monit

• smartd — scheduled self-tests, mail to root
• monit — load, ping, zpool status, filesystem fill

Network isolation

• localnet0 — isolated jail network for services
• publicnet0 — shared bridge to the LAN / WAN edge

rc.conf.network · rc.conf.network-lab

PF firewall

• block-policy return, antispoof, ICMP, SSH
• NAT: tables <jails> / <vms>
• service pf reload

Appendix: OPNsense HAProxy + ACME

Part 3 — Jails

work/bastille · VNET · bridge0

Why BastilleBSD?

• Thin layer over jail.conf + ZFS
• ZFS clones from the release snapshot
• Reusable templates for applications
• Coexists with BHYVE

Lab jails

nullfs

• data/bastille_data → /usr/local/bastille/data
• per-jail fstab, match UID/GID
• lab shortcut: bastille mount → /mnt/share

Comparison: Classic jail vs iocage vs Bastille

Issuing Wildcard

• Install acme.sh
• Configure DNS API automation
• Issue wildcard certificates
• Manage renewals

Step 1: Install acme.sh

Clone the repository and install acme.sh:

git clone https://github.com/acmesh-official/acme.sh.git
cd ~/acme.sh
./acme.sh --install -m mail@example.com

Step 2: DNS API Configuration

Browse available DNS scripts:

cd ~/acme.sh/dnsapi

Examples:

• Cloudflare
• Hetzner DNS
• Route53
• DigitalOcean
• Google Cloud DNS

Example: Hetzner DNS

Obtain an API token from:

Hetzner Console → Project → DNS Settings

Configure DNS Authentication

export HETZNER_TOKEN="<token>"

Verify Saved Configuration

View stored account settings:

cat ~/.acme.sh/account.conf

Confirm that the API token has been saved successfully.

Step 3: Certificate Authorities

Default CA

acme.sh uses:

ZeroSSL

by default.

Other supported Certificate Authorities include:

• BuyPass
• Let's Encrypt
• Various commercial CAs

Refer to the acme.sh documentation for CA selection options.

Key Algorithm Changes

Starting with acme.sh v3.0.6:

• Default key type = ec-256
• Previous default = RSA2048

If your CA requires RSA:

-k 2048

or another supported RSA key length.

Issue a Wildcard Certificate

Run:

cd ~/acme.sh

./acme.sh --issue \
  --dns dns_hetznercloud \
  -d example.com \
  -d *.example.com

DNS Validation Process

acme.sh
   ↓
DNS API
   ↓
TXT Record Created
   ↓
CA Verification
   ↓
Certificate Issued

Certificate Output

After successful issuance:

• Certificates are stored in the acme.sh directory structure
• Open certificate files using a text editor
• Deploy them to your web server, load balancer, or application

Summary

Workflow

1. Install acme.sh
2. Configure DNS API access
3. Store API token
4. Issue wildcard certificate
5. Deploy certificates
6. Configure automatic renewals

Part 4 — Examples

Reverse proxy, file sharing, and application hosting

Reverse proxy / application hosting

• Nginx or HAProxy with SSL for public-facing apps
• Nextcloud as the example application

vm-bhyve

pkg install vm-bhyve bhyve-firmware
sysrc vm_dir="zfs:work/bhyve"
vm init
vm switch create -t manual -b bridge1 public

NFS (host → VM)

• Data on data/
• NFSv4 — not VirtIO-9p for DB workloads

Samba in a jail

data → nullfs → files → SMB

OCI apps (optional)

Daemonless — Podman or AppJail · ghcr.io/daemonless/*

Appendix: 09 Daemonless

More services

Nextcloud, Zabbix, NGINX …

Part 5 — Maintenance & backups

Snapshots · config backup · replication · updates

zfsnap2

• data/home, data/media, work/bastille/jails
• zroot → use bectl instead of snapshots

Back up config

/etc, bastille, pf, monit, smartd

Updates

• Host: freebsd-update + BE
• Jails: bastille update + pkg upgrade

Part 6 — Single sign-on

Take-home appendix. Don't try this in the room.

Why bother

• One identity = one off-boarding step
• MFA in one place (Passkeys at the IdP, not per-app)
• Family/team grows without a credentials sprawl

Cost: one more critical jail. Snapshot the dataset, plan a recovery.

Pick your path

Choose one path for identity in your deployment:

Path A — LDAP-only (recommended for mixed legacy + modern services)

389-ds (LDAP)
	├─ Samba, SSH (LDAP)
	└─ apps that can speak LDAP

Path B — Keycloak-only (standalone, no LDAP required)

Keycloak (OIDC)
	└─ apps via OIDC (Nextcloud, Zabbix)

Path C — LDAP backend + OIDC layer (hybrid)

389-ds (LDAP) ─► Kanidm / Keycloak (OIDC gateway) ─► apps (OIDC)

Use Path B if you want Keycloak standalone; use Path C to support legacy LDAP consumers while offering OIDC to web apps.

The idm jail

• 10.10.0.80, VNET, on bridge0
• Datasets: data/idm (snapshot it!)
• Daemons: 389-ds-base + kanidm-server
• TLS: real LE cert via DNS-01, not self-signed

App wiring

Why not OIDC everywhere?

• Samba wants NT hashes
• NFSv4+Kerberos wants principals
• Legacy apps are forever
• Offline cache: LDAP yes, OIDC no

LDAP stays. OIDC is a layer, not a replacement.

mdo

Replacing sudo for many FreeBSD servers

• FreeBSD 15
• MAC Framework
• Privilege Delegation
• System Hardening

What Changed?

FreeBSD now includes:

• mdo(1) in base
• mac_do(4) kernel policy module
• MAC Framework integration

No additional packages required.

What mdo Actually Is

mdo is a credential transition mechanism.

mdo

Result:

user@host:~ $ mdo
root@host:~ #

mdo vs sudo

Security Model

Important Difference

mdo assumes:

SSH authentication is the security boundary.

If a user is authorized by a mac_do rule:

• They can assume the target identity
• No second password prompt exists
• SSH keys become critical

Treat mdo-enabled accounts as privileged accounts.

Enable mac_do

Add to:

/boot/loader.conf

mac_do_load="YES"

Load immediately without reboot:

kldload mac_do

Verify Module Loading

Check with:

kldstat
...
mac_do.ko
...

Without the module loaded, mdo cannot function.

Configure Rules

Rules are stored in:

/etc/sysctl.conf"

security.mac.do.rules="uid=1001>uid=0,gid=*,+gid=*"

This single line defines the policy.

Understanding the Rule

uid=1001 > uid=0,gid=*,+gid=*

Meaning:

• User 1001 may become root
• Any primary group allowed
• Any supplementary groups allowed

Equivalent conceptually to:

"Admin user may become root."

Rule Language Basics

Source Side

Who may request:

uid=1001
gid=0

Target Side

What they may become:

uid=0
gid=*
+gid=*

Multiple Rules

Example:

security.mac.do.rules="
uid=1001>uid=0,gid=*,+gid=*;
gid=0>uid=0,gid=*,+gid=*"

Allows:

• Specific admin user
• Any wheel group member

to become root.

Important Gotcha

The parser only accepts numeric IDs.

gid=0

The kernel does not resolve group names.

Fine-Grained Delegation

Allow a deploy user to become only an app account:

security.mac.do.rules="
uid=2001>uid=1500,gid=1500,+gid=1500"

Result:

• Deploy user → app user
• No root access
• No wildcard escalation

Live Rule Changes

Apply immediately:

sysctl security.mac.do.rules="..."

Verify:

sysctl security.mac.do.rules

Benefits:

• No reboot required
• Invalid rules are rejected
• Existing policy remains active

Day-to-Day Usage

Open a root shell:

mdo

Run a single command:

mdo pkg upgrade

Target another user:

mdo -u appuser

(Requires matching authorization rule.)