FreeBSD – Configure a private IP jail

If you use jails (or want to use jails) but your pool of IP addresses is somewhat limited don’t worry. You can fully configure and use a jail in a private IP, and even assign port forwarding from the “outside” network to reach the jail.

First things first, create a loopback interface clone and assign it an IP address:

ifconfig lo1 create
ifconfig lo1 inet

To make this live across reboots add the following lines to /etc/rc.conf:

ifconfig_lo1="inet netmask 0xffffffff"

Now, use ezjail to create and configure a new jail and assign this internal IP address. If you start the jail now you will be able to access it, but in the jail itself you will not be able to access the outside world… this is where NAT comes in.

There is at least 2 options, the natd daemon + ipfw or the pf route. I opted for the pf route simply because the configuration is much more simple (but if you are more pro-efficient with natd and ipfw probably it’s the best bet).

As always be careful when messing with a firewall, specially if you are working on a remote server, as you can lock yourself out of your own server. I usually set up an at job that reboots to the previous state in half an hour or so to test everything before committing the changes permanently to rc.conf (to start and stop services with no rc.conf entry you can use the onestart/onestop option).

This is the most economical version of /etc/pf.conf (adjust the external interface and the jail IP (the first two lines):


set skip on lo0
scrub in all

nat on $ext_if from lo1:network to any -> $ext_if

pass all

and fire up pf

service pf start

and now from inside the jail you can access the world. Actually, the FreeBSD manual (in it’s current writing) states an additional step, that is to enable the sysctl gateway_enable=”YES” option to nat work, but I didn’t enable it on two machines running FreeBSD 10 and is working perfectly. In set-ups with natd + ipfw you have to enable it for sure, on old FreeBSD versions with pf I just don’t know… but if you can’t access the world from within the jail enable this would be on top of my list.

To make this permanently just have to add to /etc/rc.conf


Now, that you have the jail all set-up, It’s about time to expose a service to the world (let’s say for example a HTTP server running clear and ssl – ports 80 and 443), you just need a tweak in /etc/pf.conf:


set skip on lo0
scrub in all

nat on $ext_if from lo1:network to any -> $ext_if
rdr pass on $ext_if proto tcp from any to $ext_if port $PORT_WWW -> $JAIL_SRV

pass all

You can jail services without using external IPs, assign HDD space via ZFS or virtual disk files, set CPU core(s) affinity, or fine grained memory and CPU limits via rctl.

Pretty cool!


If you are using FreeBSD jails, or are planning to use (just make sure it’s the right solution as there are some caveats), you should look into ezjail, a powerful set of jail management scripts. It can really help save time and do things better: more secure jails, less space usage, easier upgrades, etc…

Install from ports (that was easy and predictable)

cd /usr/ports/sysutils/ezjail/ 
make install clean

Then, configure ezjail behaviour at /usr/local/etc/ezjail.conf, the most important thing is to setup the ezjail directory, where all the jails will be written to (moving afterwards is a pain in the ass), so consider your partitions and backups plan when setting this up.

Next thing is to build world and make basejail. Just one command:

ezjail-admin update -b

If you don’t have src, this is the error you will get:
Error: Your source tree in /usr/src seems to be incomplete (Makefile is missing).

Very easy to get src, type ‘sysinstall’ go to Configure, Distributions, src, All (everything gets selected), Exit, Exit, FTP, select a FTP site, YES, and there a couple of minutes later (your mileage may vary) /usr/src is populated with everything you need. Then exit sysintall and repeat the command above.

If you have already a built world (ex: from a system upgrade) just need to install it

ezjail-admin update -i

Now, everything is set to to go!

To create a new jail

ezjail-admin create hostname ip-address

With a 100G file-Backed md filesystem

ezjail-admin create -i -s 100G hostname ip-address

To list jails managed by ezjail

ezjail-admin list

List all running jails (in ezjail scope and others) you can fallback to the normal system command


To get a console inside a runing jail

ezjail-admin console hostname

WARNING! You don’t actually get a tty, so some things work strange, for instance: ssh and sftp to remote machines, mysql imports showing password as you type, etc… if something works strange, to be safe better log in through ssh, you have been warned.

Start/stop all jails

/usr/local/etc/rc.d/ start 
/usr/local/etc/rc.d/ stop

Star/stop one jail

/usr/local/etc/rc.d/ start hostname 
/usr/local/etc/rc.d/ stop hostname

To check the jail status, if it is running or not use the jls command


To start automatic at boot just set the ezjail_enable in /etc/rc.conf. It will run all jails in ezjail scope, except if you specify some jail not to boot automatically:

ezjail-admin config -r norun hostname

of course if you change your mind, you can simply revert it

ezjail-admin config -r norun hostname

About the ports, as i am very happy with the current setup, and it seemed a waste of space to have duplicate ports tree in the same machine… first i simply monted null_fs my existing ports tree of the main host into /ezjail_dir/basejail/usr/ports but i couldn’t access it from within the jails… probably double mount_null problem, as the basejail dir is also null mounted, so i add it to the fstab of each jail. Just look and edit /etc/fstab.jailname and:

/usr/ports /usr/jails/jaildir/usr/ports nullfs ro 0 0

Also tweak main host and jails /etc/make.conf to avoid any interference with ports building, files and indexes.


Don’t forget to look into /usr/local/etc/ezjail/ directory, where all the configs for each jail live in separate files. I find it much easier to change or setup things like cpusets and multiple ip here than thru ezjail-admin command.

With a multi core CPU, you can to set a jail cpu affinity to use one particular core, just go to the /usr/local/etc/ezjail directory, then find and open your jail configuration file, should be something like jailhostname_com. Edit the line  export jail_iedp_pt_cpuset=”” and set the core you want to assign the jail to.

To get the number of cpu/cores just type as root

sysctl hw.model hw.ncpu

At last, if you want to delete a jail simply type (and say goodbye)

ezjail-admin delete -w hostname

If you are using file based disks (md disks) and need to check them just stop the jail (if it is running). Attach the disk, use fsck, and detach it:

mdconfig -a -f file.img
fsck -t UFS /dev/mdxx
mdconfig -d -u xx

(replace the xx by the number outputed in the first command).

Updating the jails OS copy after a freebsd-update upgrade is also very easy. You must take note the original OS version and the target upgrade version. So upgrading from 10.0 to 10.1 you must always keep this information.


First in the main host do the freebsd-update upgrade operations

# freebsd-update upgrade -r 10.1-RELEASE

Nnormally includes automatic file adds/edits/deletes, manual rebooting and running freebsd-update install afterwards to finish.

# freebsd-version

The main host is at 10.1, but your jails are out of sync in 10.0, again with ezjail it’s a breeze to fix this, just issue this command (note the -s parameter is the original OS version):

# ezjail-admin update -U -s10.0-RELEASE

There you go, a very simple guide to ezjail jails management, as always feel free to ask any questions or add up some knowledge. Also, try and make yourself comfortable before stepping into production, and remember if all goes terrible bad you are on your own.