FreeBSD Jails For Fun and Profit
Posted on 16th of November 2020Docker has recently stormed into software
development. While its concepts are powerful and valuable, similar
tools have been used in systems for decades. FreeBSD’s
jails in one of
those tools which build upon even older
chroot(2).
To put it shortly, with these tools, you can make a safe environment
separated from the rest of the system.
Jails in FreeBSD are by no means a new tool (introduced in 4.x), but
for one reason or another, I haven’t used them that often, which is a
shame since they are so powerful. So I wanted to explore this concept
in a concise and summarized manner.
Templates
ZFS datasets are a great way of creating templates for jails since,
after the template creation, you can easily create new jails with zfs clone or zfs send/receive. Typically, people divide jails to
complete and service jails, where the former resembles a real FreeBSD
system, and the latter is often dedicated to
applications/services. I’ll cover complete jails for now.
Creating templates starts with creating a dataset for your jail and
template. Here I’ll make a new dataset for the base installation of
FreeBSD 12.2.
$ sudo zfs create -o mountpoint=/vm zroot/vm
$ sudo zfs create zroot/vm/tmpl
$ sudo zfs create zroot/vm/tmpl/12.2
After that, fetch the base installation itself:
$ fetch ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/12.2-RELEASE/base.txz
# Fetch all the necessary stuff for your template, e.g. lib32 if needed
$ sudo tar -xJvpf base.txz -C /vm/tmpl/12.2
After that, you should write a minimum viable /etc/rc.conf for the
template:
$ sudo emacs /vm/tmpl/12.2/etc/rc.conf
# Start or stop services
sendmail_enable="NO"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"
syslogd_flags="-ss"
cron_flags="-J 60"
You can also disable some unnecessary jobs for jails:
$ sudo emacs /vm/tmpl/12.2/etc/periodic.conf
# No output for successful script runs.
daily_show_success="NO"
weekly_show_success="NO"
monthly_show_success="NO"
security_show_success="NO"
# Output to log files which are rotated by default.
daily_output="/var/log/daily.log"
daily_status_security_output="/var/log/daily.log"
weekly_output="/var/log/weekly.log"
weekly_status_security_output="/var/log/weekly.log"
monthly_output="/var/log/monthly.log"
monthly_status_security_output="/var/log/monthly.log"
# No need for those without sendmail
daily_clean_hoststat_enable="NO"
daily_status_mail_rejects_enable="NO"
daily_status_mailq_enable="NO"
daily_queuerun_enable="NO"
# Host does those
daily_status_disks_enable="NO"
daily_status_zfs_zpool_list_enable="NO"
daily_status_network_enable="NO"
daily_status_uptime_enable="NO"
daily_ntpd_leapfile_enable="NO"
weekly_locate_enable="NO"
weekly_whatis_enable="NO"
security_status_chksetuid_enable="NO"
security_status_neggrpperm_enable="NO"
security_status_chkuid0_enable="NO"
security_status_ipfwdenied_enable="NO"
security_status_ipfdenied_enable="NO"
security_status_ipfwlimit_enable="NO"
security_status_ipf6denied_enable="NO"
security_status_tcpwrap_enable="NO"
You also might want to enable ports in your jail:
$ sudo mkdir /vm/tmpl/12.2/usr/ports
$ sudo mkdir -p /vm/tmpl/12.2/var/ports/{distfiles,packages}
$ sudo emacs /vm/tmpl/12.2/etc/make.confWRKDIRPREFIX = /var/ports
DISTDIR = /var/ports/distfiles
PACKAGES = /var/ports/packages
Apply system updates to the template:
$ sudo freebsd-update -b /vm/tmpl/12.2 fetch install
Lastly, take a snapshot:
Strictly speaking, a template is a snapshot, not a
dataset. The snapshot can be cloned or sent/received to generate
new datasets for production jails.
$ sudo zfs snapshot zroot/vm/tmpl/12.2@complete
This creates a snapshot of zroot/vm/tmpl/12.2 named complete. You
can then check your current snapshots with the following:
$ sudo zfs list -t snapshot
Creating jails from the template
Now you should create a new jail based on that snapshot. You can do it
either with zfs clone or zfs send/receive:
Difference Between the Two
“A clone is a writable volume or file system whose initial contents
are the same as the dataset from which it was created. As with
snapshots, creating a clone is nearly instantaneous and initially
consumes no additional disk space. In addition, you can snapshot a
clone.” [1]
“The zfs send command creates a stream representation of a snapshot
that is written to standard output. By default, a full stream is
generated. You can redirect the output to a file or to a different
system. The zfs receive command creates a snapshot whose contents
are specified in the stream that is provided on standard input. If a
full stream is received, a new file system is created as well. You
can send ZFS snapshot data and receive ZFS snapshot data and file
systems with these commands. See the examples in the next section.”
[2]
$ sudo zfs clone zroot/vm/tmpl/12.2@complete zroot/vm/jail1
# OR
$ sudo sh -c "zfs send zroot/vm/tmpl/12.2@complete | zfs receive zroot/vm/jail1"
Jail configurations
# /etc/rc.conf
cloned_interfaces="lo0"
# PF is used for NAT and port forwarding.
pf_enable="YES"
pflog_enable="YES"
jail_enable="YES"
jail_list="jail1"
## /etc/jail.conf
exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
exec.clean;
mount.devfs;
host.hostname = $name;
path = "/vm/$name";
exec.consolelog = "/var/log/jail_${name}_console.log";
exec.prestart = "cp /etc/resolv.conf $path/etc";
exec.poststop = "rm $path/etc/resolv.conf";
jail1 {
ip4.addr = "lo0|127.1.1.1/32";
ip6.addr = "lo0|fd00:1:1:1::1/64";
allow.chflags;
allow.raw_sockets;
}# /etc/hosts
...
127.1.1.1 jail1
fd00:1:1:1::1 jail1
Jail management
FreeBSD provides nifty built-in tools for jail management:
Start all jails.
$ sudo service jail start
Start a specific jail(s).
$ sudo service jail start jail1
Log in to jail.
$ sudo jexec jail1
Run a command on a jail.
$ sudo jexec jail1 ifconfig
List running jails.
$ jls
$ jls -v
$ jls -s
So that’s how you can spin up a simple restricted environment on your
FreeBSD system. Of course, this topic still has many things to cover,
e.g., in-depth networking and configurations.
Notes
References