My Qmail installation guide reloaded

Qmail ReloadedA couple of years ago I posted my Qmail installation guide, and has expected it served me good when was time to reform to the old mail server. But, i made some changes on this iteration and i think is more polished and shiny than ever.

Again, this is to my own reference, but i will be very glad if it also can help someone. On the other hand, if you follow it, and nukes your system or kills every life form on Earth please don’t blame me. You are warned.

The old picture:


2 Qmail instances, 1 published MX record that accepts emails from other MTAs, does the RBL checks and forwards the passed emails to the main Qmail instance via artificial smtproutes. The forwarded emails are then checked against virus (by Clamav) and spam (by SpamAssassin) trough qmail-scanner qmail-queue drop in replacement.

Users receive and send email trough the non published MX Qmail instance. They need to smtp-auth to relay email (send email to remote domains). Delivery to local domains doesn’t require smtp-auth.

Identified problems:

1 – One problem is that the main Qmail instance (that has no published MX records), that works with Vpopmail and holds all the accounts information, maildirs and email is somehow vulnerable:

The main weakness of this installation, is that if a clever spammer discovers that accepts incoming emails for local domains, he can spam down your users bypassing the rbl tests.

also, one has to rememeber that has SPF and A records published, and it’s IP is printed on all outgoing email headers, so it’s not anonymous.

2 – The user debug is somewhat tricky, if there is a smtp-auth client configuration problem. The problem is that the user will be able to send emails to local domains, but will get the dreadful 553 sorry that domain isn’t in my list of allowed rcpthosts (#5.7.1) error.

3 – Qmail-scanner, is a very neat piece of software, but it is fundamentally flawed performance wise because for each and every email it must load the PERL interpreter.

4 – Restarting Qmail every 15m to recognize new or deleted domains is plain dumb.

The new picture:


All of the previous mentioned issues have been addressed and polished. The main Qmail instance (mail) will only accept outside authenticated connections for both local and remote deliveries. The external email comes trough the published mx record Qmail instance only, filtered by rbl, then routed to Amavis for virus and spam scans, and finally routed to the main Qmail instance (if virus and spam free). In this scenario you must trust your customers, because as they authenticate and send emails, these will bypass all the virus and spam checks.

Let’s put our hands to work, the first slice is on point 15 of the original guide “Clam Anti Virus, Spam Assassin and Qmail-scanner”, this version will move the virus and spam filter to the other Qmail instance. So follow the original guide until point 15, and then:

1 – Install qfilter

cd /usr/ports/mail/qmail-qfilter/
make install clean

2 – Make a shell script wrapper that will invoke the filters used by qfilter

mkdir -p /var/qmail/qfilter
edit /var/qmail/qfilter/qfilter-wrapper

and put these contents on the file

exec /usr/local/bin/qmail-qfilter /var/qmail/qfilter/smtp-auth-only

save and mark it executable

chmod +x /var/qmail/qfilter/smtp-auth-only

actually there is only one filter being invoked (smtp-auth-only), but qfilter supports several filters (exec /usr/local/bin/qmail-qfilter /path/to/filter-one –/path/to/filter-two –/path/to/filter-three)

3 – Install the smtp-auth-only filter

This is just a very simple perl script that will test the presence of the environment variable TCPREMOTEINFO, as this variable is only set upon successful smtp-auth. If the mail comes from an authenticated user the script returns 0, else if it’s from a non-authenticated user the script returns 31 signaling a permanent error.

edit /var/qmail/qfilter/smtp-auth-only

the script is very simple


if (defined $ENV{'TCPREMOTEINFO'} == false) {
        use Sys::Syslog qw(:DEFAULT :standard);
        openlog("qfilter", 'ndelay,pid', 'mail');
        syslog('info', "No SMTP-Auth - Rejecting Email");
        exit 31;

exit 0;

save it and mark it executable

chmod +x /var/qmail/qfilter/smtp-auth-only

4 – Adjust /etc/tcp.smtp to use qfilter

this is my last line now of /etc/tcp.smtp


it accepts connections from everywhere (if cpu load > 20 rejects connections) it bypasses SPF and RBL checks, and it uses qfilter-wrapper as qmailqueue. After

qmailctl cdb

to build the new smtp tcp rules cdb file and reload qmail, the main Qmail instance will only accept authenticated user email. Email routed from mx should match a previous /etc/tcp.smtp rule.

5 – Install Clam Anti Virus, Spam Assassin and Amavis

This step kind of mimics the step 15 on the original guide, with two main differences. We are installing all the filtering software on the MX instance of Qmail. And qmail-scanner as been replaced by Amavis.

So, log in to the MX console and install the software.

cd /usr/ports/security/clamav
make install clean


cd /usr/ports/mail/spamassassin
make install clean


cd /usr/ports/security/amavisd-new
make install clean


as always on FreeBSD the installation is easy and a breeze. Now the fun part, configuring and make all this work together…

6 – Configure Clam Anti Virus

First ClamAV and FreshClam (the anti-virus updater daemon). Here’s a comment striped out of /usr/local/etc/clamd.conf

LogSyslog yes
LogFacility LOG_MAIL
LogVerbose yes
ExtendedDetectionInfo yes
PidFile /var/run/clamav/
DatabaseDirectory /var/db/clamav
LocalSocket /var/run/clamav/clamd.sock
FixStaleSocket yes
ReadTimeout 300
CommandReadTimeout 5
User vscan
AllowSupplementaryGroups yes
ScanMail yes

and the comment stripped version of /usr/local/etc/freshclam.conf

DatabaseDirectory /var/db/clamav
LogVerbose yes
LogSyslog yes
LogFacility LOG_MAIL
PidFile /var/run/clamav/
DatabaseOwner vscan
AllowSupplementaryGroups yes
NotifyClamd /usr/local/etc/clamd.conf

There are few modifications to the distribution configuration files, mainly 2 things, to run clamd/freshclam daemons as the user ‘vscan’, the same user that will run amavis, and to log via syslog mail facility.

It makes perfect sense to take advantage of syslog and newsyslog automatic maintenance and log rotation. Also, having most of stuff logging to /var/log/mail makes it easy to spot any error message outputted by any of the several components. The downsize, is that in a busy server the log can become a bit messy.

Adjust the ownership on ClamAV directories:

chown -R vscan:vscan /var/db/clamav
chown -R vscan:vscan /var/run/clamav

add the rcvars to /etc/rc.conf

and start both of the daemons

/usr/local/etc/rc.d/clamav-clamd start
/usr/local/etc/rc.d/clamav-freshclam start

7 – Configure Spamassassin

First, as Spamassassin uses the GeoIP database, you should have an updated database on /usr/local/share/GeoIP/GeoIP.dat, to do so automaticaly write this file on /usr/local/etc/periodic/daily/updategeoip


cd /usr/local/share/GeoIP
/usr/local/bin/wget -q
gzip -d -f GeoIP.dat.gz

exit 0

and mark it executable

chow +x /usr/local/etc/periodic/daily/updategeoip

and run it manually (you should have installed on your system /usr/ports/ftp/wget)


update and compile Spamassassin rules


and make this process automatic, edit /usr/local/etc/periodic/weekly/spamassassin

#! /bin/sh

/usr/local/bin/sa-update && /usr/local/bin/sa-compile

exit 0

mark it executable, and run by hand the first time

chmod +x /usr/local/etc/periodic/weekly/spamassassin

Spamassassin doesn’t need so much configuration, and it pretty much works out of the box, but i made some fine tuning to everything play happy, so there it is the commented striped version of /usr/local/etc/mail/spamassassin/

use_dcc 1
dcc_home /var/dcc
dcc_path /usr/local/bin/dccproc
dcc_timeout     10
add_header all  DCC _DCCB_: _DCCR_
use_pyzor 1
pyzor_path /usr/local/bin/pyzor
use_razor2 1
razor_config /var/amavis/.razor/razor-agent.conf
score RAZOR2_CHECK 2.500
score PYZOR_CHECK 2.500
score DCC_CHECK 4.000

create the /var/amavis/.razor directory, and set up razor

mkdir /var/amavis/.razor
razor-admin -home=/var/amavis/.razor -create
razor-admin -home=/var/amavis/.razor -discover

and change ownership to the vscan user

chown -R vscan:vscan /var/amavis/.razor

time to set up the rc vars at /etc/rc.conf and start Spamassassin (replace aaa.bbb.ccc.ddd for the allowed IP address to connect)


and start it

/usr/local/etc/rc.d/sa-spamd start

8 – Configure Amavis

Amavis will be the glue between Qmail and ClamAV and Spamassassin in a dual MTA setup. It will accept routed emails from Qmail (mx instance) on port 10024, fiter, and re-route to the main Qmail for local delivery (email instance).

Here it is the /usr/local/etc/amavisd.conf configuration file. Now some customizations required to amavis work properly:

  • set $mydomain and $myhostname to your host fqdn
  • configure $forward_method = ‘smtp:[aaa.bbb.ccc.ddd]:25’; Set aaa.bbb.ccc.ddd to the IP address of the main Qmail instance (email host) to where the filtered emails are forward. Remember that in the Qmail instance you need a corresponding entry in /etc/tcp.smtp that accepts the forward emails and skips SPF and RBL checks (replace aaa.bbb.ccc.ddd for the incoming IP of Amavis):
  • the $max_servers should, as commented, match the width of your MTA pipe /var/qmail/control/concurrencylocal
  • @local_domains_maps = [‘.’]; we accept every incoming email as a local domain email, because by configuration the mx Qmail instance will only accept and forward to Amavis emails to local domains
  • customize $inet_socket_bind, generally the loopback address IP should be fine, but if your are running inside a jail (and if you are following this guide you are) replace the loopback IP for the main jail IP
  • setup @inet_acl list (this is space delimited list of IPs that Amavis will accept email from). If you are running everything in the same jail (Qmail mx instance and Amavis) this is the main jail IP, if Qmail mx is running in other jail or host add the mx Qmail outgoing IP

These are some of the most important things that you should consider to setup Amavis to your own taste, and as pretty neat software everything (or just about everything) is customizable. The configuration file has extensive comments so it’s easy to understand each and every option:

  • In this setup Amavis is logging to syslog mail facility, $DO_SYSLOG = 1; and $SYSLOG_LEVEL = ‘’; scroll up to find out why. You can change this to a $LOGFILE. Also setup the $log_level
  • Virus, banned and spam (after $sa_kill_level_deflt threshold) emails are plain discarded, bounce only in case of bad headers.
    $final_virus_destiny = D_DISCARD; # (defaults to D_DISCARD)
    $final_banned_destiny = D_DISCARD; # (defaults to D_BOUNCE)
    $final_spam_destiny = D_DISCARD; # (defaults to D_BOUNCE)
    $final_bad_header_destiny = D_BOUNCE; # (defaults to D_PASS), D_BOUNCE suggested
  • you can customize $virus_admin and $spam_admin with a email address to receive reports when virus/spam email is detected, in this case you should also configure the from addresses in $mailfrom_notify_admin, $mailfrom_notify_recip, $mailfrom_notify_spamadmin
  • this configuration example does not notify me of positives, but i keep them in a quarantine dir, so i can do postmortem analysis and recovery,
    $QUARANTINEDIR = ‘/var/virusmails’;
    # Separate quarantine subdirectories virus, spam, banned and badh within
    # the directory $QUARANTINEDIR may be specified by the following settings
    # (the subdirectories need to exist – must be created manually):
    $virus_quarantine_method = ‘local:virus/virus-%i-%n’;
    $spam_quarantine_method = ‘local:spam/spam-%b-%i-%n’;
    $banned_files_quarantine_method = ‘local:banned/banned-%i-%n’;
    $bad_header_quarantine_method = ‘local:badh/badh-%i-%n’;
  • you can also customize the spam score required to each action
    $sa_tag_level_deflt = undef; # always add spam info headers
    $sa_tag2_level_deflt = 5.0; # subject will be re-written with $sa_spam_subject_tag value
    $sa_kill_level_deflt = 10; # email will not be delivered, and we keep a copy in quarantine
    $sa_dsn_cutoff_level = 15; # Since we are using D_DISCARD, this setting will serve no purpose, but if you were using D_BOUNCE, you can use this to set a level at which the sender will no longer be notified

and many more options that you can/should look into. If you are going to quarantine emails you should create the quarantine directories:

mkdir -p /var/virusmails/badh/
mkdir -p /var/virusmails/banned/
mkdir -p /var/virusmails/spam/
mkdir -p /var/virusmails/virus/

chown -R vscan:vscan /var/virusmails

also, it’s not a bad idea to put a line in root cron to delete older (30 days older) quarantined emails:

crontab -e

05 05 * * * /usr/bin/find /var/virusmails/* -type f -mtime +30 -exec /bin/rm -f {} \;

Finally! add the rc var at /etc/rc.conf


and start it

/usr/local/etc/rc.d/amavisd start

9 – Configure Qmail to use Amavis

Just a simple php script run every 10 minutes by cron will take care of this. As a bonus when you add, rename or delete a domain the Qmail mx instance will pick up the changes.

Edit /var/qmail/control/make_smtp_routes and adjust aaa.bbb.ccc.dd with the Amavis listening IP:port ($inet_socket_bind in amavisd.conf):

#! /usr/local/bin/php

$smtp_route = 'aaa.bbb.ccc.ddd:10024';

$rcpthosts     = file('/var/qmail/control/rcpthosts');
$morercpthosts = file('/var/qmail/control/morercpthosts');

$hosts = array_merge($rcpthosts, $morercpthosts);
$hosts = array_filter($hosts);

$fp = fopen("/var/qmail/control/smtproutes.tmp", "w");
foreach ($hosts as $host)
    fwrite($fp, trim($host).":".$smtp_route."\n");

if (md5_file('/var/qmail/control/smtproutes.tmp') == md5_file('/var/qmail/control/smtproutes')) {

syslog(LOG_INFO, "New /var/qmail/control/smtproutes");

rename("/var/qmail/control/smtproutes.tmp", "/var/qmail/control/smtproutes");

syslog(LOG_INFO, "Restarting Qmail");
exec('/root/bin/qmailctl restart');



mark it executable

chown +x /var/qmail/control/make_smtp_routes

and add it to cron

cron -e
*/10 * * * * /var/qmail/control/make_smtp_routes > /dev/null 2>&1

That’s it, this is the end. Now go grab a well deserved beer and behold your brand new system.

Final toughts

The system is cool, addressed the issues of the old system and is maintenance free. But, there is some space to improvements:
– develop an API (work in progress) that allows for administration, domain management and email management of the system. With this piece in place is then easy to integrate and develop admin and control panels that replace the outdated qmailadmin panel and administrative tasks on the command line.
– related with the API, to give domain managers the possibility to fine tune per domain anti-virus, spam, quarantine and notification settings. This also implies a deeper knowledge of Amavis configuration.
– to compile a complete and comprehensive guide that incorporates the original guide and the stuff on this one.


FreeBSD – Configure a private IP jail

If you use jails (or want to use jails) but your pool of IP addresses is somewhat limited don’t worry. You can fully configure and use a jail in a private IP, and even assign port forwarding from the “outside” network to reach the jail.

First things first, create a loopback interface clone and assign it an IP address:

ifconfig lo1 create
ifconfig lo1 inet

To make this live across reboots add the following lines to /etc/rc.conf:

ifconfig_lo1="inet netmask 0xffffffff"

Now, use ezjail to create and configure a new jail and assign this internal IP address. If you start the jail now you will be able to access it, but in the jail itself you will not be able to access the outside world… this is where NAT comes in.

There is at least 2 options, the natd daemon + ipfw or the pf route. I opted for the pf route simply because the configuration is much more simple (but if you are more pro-efficient with natd and ipfw probably it’s the best bet).

As always be careful when messing with a firewall, specially if you are working on a remote server, as you can lock yourself out of your own server. I usually set up an at job that reboots to the previous state in half an hour or so to test everything before committing the changes permanently to rc.conf (to start and stop services with no rc.conf entry you can use the onestart/onestop option).

This is the most economical version of /etc/pf.conf (adjust the external interface and the jail IP (the first two lines):


set skip on lo0
scrub in all

nat on $ext_if from lo1:network to any -> $ext_if

pass all

and fire up pf

service pf start

and now from inside the jail you can access the world. Actually, the FreeBSD manual (in it’s current writing) states an additional step, that is to enable the sysctl gateway_enable=”YES” option to nat work, but I didn’t enable it on two machines running FreeBSD 10 and is working perfectly. In set-ups with natd + ipfw you have to enable it for sure, on old FreeBSD versions with pf I just don’t know… but if you can’t access the world from within the jail enable this would be on top of my list.

To make this permanently just have to add to /etc/rc.conf


Now, that you have the jail all set-up, It’s about time to expose a service to the world (let’s say for example a HTTP server running clear and ssl – ports 80 and 443), you just need a tweak in /etc/pf.conf:


set skip on lo0
scrub in all

nat on $ext_if from lo1:network to any -> $ext_if
rdr pass on $ext_if proto tcp from any to $ext_if port $PORT_WWW -> $JAIL_SRV

pass all

You can jail services without using external IPs, assign HDD space via ZFS or virtual disk files, set CPU core(s) affinity, or fine grained memory and CPU limits via rctl.

Pretty cool!

Setting up SSH access with server keys (no password) in FreeBSD

You want to login from user@host-a to user@host-b automatically (with no password prompt).

Install ssh-copy-id in the host-a. This little handy script takes care of all the dirty details from your back.

cd /usr/ports/security/ssh-copy-id
make install clean

then run (still in host-a)

ssh-copy-id user@host-b

If you get the error “ERROR: No identities found” then you must generate your keys first and then re-run ssh-copy-id. To generate a pair of private/public SSH keys you must issue:

mkdir ~/.ssh
chmod 700 ~/.ssh
ssh-keygen -t rsa

You can leave the pass-phrase empty for automatic logins (no password prompt) or the more secure but less practical password prompt each and every time that you use the newly generated keys.

Next login from host-a to user@host-b will be made with SSH keys.

The dirty details

  • the user@host-a public key, usually ~/.ssh/ is copied to host-b user/.ssh/authorized_keys
  • the host-b public host key (/etc/ssh/ is copied to host-a known hosts

FreeBSD migrating user accounts

FreeBSDQuick and dirty way to migrate user accounts from one FreeBSD box to another. From the source box you will only need two files:


copy them to the target box, but DON’T overwrite (yet) the existing files there. Put them in /root/master.passwd and /root/group. Compare the copied files with the existing ones for new system users that may exist. If there are any new users/groups add them to the copied files.

Then move /root/group to /etc/group and run this magical command

pwd_mkdb -p /root/master.passwd

It will install in /etc/master.passd and recreate all the needed files (/etc/pwd.db, /etc/spwd.db and /etc/passwd).

Securing SSH with SSHGuard

SSHGuardIf you have a remote server running some flavor of Unix or Linux 99% chances that you use SSH. The best security practice is to use an access key with password and disable password access altogether. But you end up loosing some flexibility (for some customers Putty is this utterly complex piece of software, imagine them playing with SSH keys…).

The best you can do is to enforce a better user password policy, but even so, as every password service it’s at mercy of brute force attacks. These attacks consume precious clock cycles and worst case scenario they can break a password and gain access to the system.

So, here comes SSHGuard to our rescue. It’s a pretty neat piece of software that is highly flexible and customizable to ones system, needs and paranoia level. On top of that is maintenance free and very easy to setup.

I’m using FreeBSD and the venerable (yet, very capable) IPFW firewall. The choice of the firewall is simply because it’s the one that i am more pro-efficient with.

First thing is to enable IPFW on your system. Open /etc/rc.conf and add these lines


actually this setup is only to bring IPFW up, it doesn’t filter anything, all the traffic is passed trough. But if you forget the firewall_type=”open” rule and start the firewall you will be lock out, because the default is no traffic allowed… (and you win a drive to the data-center or some kind of remote rescue shell procedure).

Start IPFW

/etc/rc.d/ipfw start

and check that is running

ipfw show

Now, you are ready to install SSHGuard itself, very easy task

cd /usr/ports/security/sshguard-ipfw
make install clean

and enable it in /etc/rc.conf


Ready? Start it

/usr/local/etc/rc.d/sshguard start

Still, there is a final thing to take care. SSHGuard uses syslogd to monitor incoming (failed) logins. So, you must edit /etc/syslog.conf and uncomment (or add if it’s not there) the line that the SSHGuard port added.;     |exec /usr/local/sbin/sshguard

And restart syslogd

/etc/rc.d/syslogd restart

And now your SSH service should be bullet proof to brute force attacks. Keep safe!

UPDATE 2014-02-23

Latest versions of SSHGuard don’t use syslogd any more, it uses an internal “log sucker” that follows the logs. The default logs are “/var/log/auth.log:/var/log/maillog”, as I don’t want it to follow /var/log/mailog i override this in /etc/rc.conf with:


UPDATE 2014-08-05

For several reasons, I have switched from IPFW to PF. So the port to install is /usr/ports/security/sshguard-pf/ and you must add this line to your /etc/pf.conf and enable PF in /etc/rc.conf

table  persist

then to list the blocked IPs

pfctl -t sshguard -T show

to remove an IP from the list

pfctl -t sshguard -T delete aaa.bbb.ccc.ddd

to remove all the IPs

pfctl -t sshguard -T flush


Also going to test drive on a debian box fail2ban, and will soon post quick review and differences, drawbacks, benefits versus sshguard.