Procmail with Qmail + Vpopmail

Following the qmail threads in this blog, and after a successful experience filtering emails in the server with a php script! time was upon to thinker a bit with the elder of all email server filters, Procmail – the mail processing utility for Unix.

Whe are talking really old stuff, as Wikipedia states the initial release in 1990, so 26 years from this writing, and about a zillion years in computer time (a date so old that it’s closer to the Unix Epoch than it is of today).

As usual, the easy part in FreeBSD is the installation:

cd /usr/ports/mail/procmail
make install

there. Now the tricky part, that is to make it play nicely with Qmail+Vpopmail setup. For the first experiences you probably should setup a couple of test accounts.

The concept is pretty simple, for an account that you want to filter email with Procmail we are going to add/or change the .qmail file that controls the email delivery to a filter script that invokes procmail and throw back a proper qmail exit code according to Procmail result.

It is very important to take into account that in this setup Procmail DOES NOT deliver the email directly, it filters the email, and according with the recipes rules, it can stop the delivery chain, forward the email, invoke an external command, etc.

Without further delays. The customized .qmail that calls the filter script:

| /home/vpopmail/domains/mydomain.com/teste/procmail_filter
/usr/home/vpopmail/domains/mydomain.com/teste/Maildir/

this is pretty simple and standard stuff. The qmail-local parses the .qmail file. The line with a pipe means to feed the message to the specified program. The command is invoked by qmail-command that runs sh -c command in the home directory, makes the email messsage available on standard input and setups some environment variables.

For this thread the most important stuff are the exit codes:

Command’s exit codes are interpreted as follows:
0 means that the delivery was successful;
99 means that the delivery was successful, but that qmail-local should ignore all further delivery instructions;
100 means that the delivery failed permanently (hard error);
111 means that the delivery failed but should be tried again in a little while (soft error).
Currently 64, 65, 70, 76, 77, 78, and 112 are considered hard errors, and all other codes are considered soft errors, but command should avoid relying on this.

With this info it’s pretty straight forward to devise the procmail_filter script.

#!/bin/sh

/var/qmail/bin/preline /usr/local/bin/procmail ./.procmailrc

status=$?

if [ $status -eq 99 ]
then
  status=99
elif [ $status -eq 0 ]
then
  status=0
elif [ $status -le 77 ]
then
  status=111
else
  status=100
fi

exit $status

and mark it executable.

The first line (after the shebang) the script calls qmail preline program, it simply inserts at the top of each message a UUCP-style From_ line, a Return-Path line, and a Delivered-To line because Procmail doesn’t understand the qmail environment variables. Calls procmail and sets Procmail configuration file the .procmailrc in the same directory.

Now for the .procmailrc stripped down to a very simple example:

SHELL = /bin/sh
LOGFILE=./pm.log
LOG="
"
VERBOSE=yes

# Exitcodes
# 0 normal delivery
# 99 silent discard a message
# 100 bounce a message

# Recipes

:0
* ^From: email@domain.com
{
  EXITCODE=99

  :0
  | /usr/local/libexec/dovecot/deliver -d delivertothis@email.com

}

# Avoid duplicates
:0
/dev/null

The top lines are the configuration variables, I would strongly suggest to use a log file for testing, in this example called pm.log that lives in the same directory. The strange LOG directive simple adds a new line to each log entry.

Then the recipes, in this example we match all emails from email@domain.com and deliver it to a local email delivertothis@email.com using Dovecot deliver command (so it takes care of Maildir quota), and set exitcode 99 to discard the message. Exit code 99 means that the delivery was successful, so all .qmail further delivery instructions will be ignored.

We could simply put an email address for a external email address, or even a local address but with less efficiency as it will trigger a new delivery process. This is auto-magical thanks to FreeBSD mailer.conf wrapper.

:0
* ^From: email@domain.com
{
  EXITCODE=99

  :0
  delivertothis@email.com

}

The last lines avoid duplicates, Procmail /dev/nulls the message and gives back the delivery control to the .qmail flow.

That’s it, everything playing nice with each other in old good UNIX tradition.

.qmail delivery filtering by adress and subject

My goal was simple, to filter incoming emails by subject and to address (on a catch-all address…) in the server and move the matches to a specific folder instead of the normal delivery. I’m running a Qmail+Vpopmail system, and these directions should be valid for similar setups.

Probably i could do this with Procmail or Maildrop, but it seemed all so complicated to do just a simple one time task that i opted in for the fun route, doing it myself… for a recurrent email filtering task, multiple accounts, customization, any kind of heavy email filtering i strongly suggest to stop here and go read about Procmail or Maildrop.

Still here? Good. First, add a .qmail (see dot.qmail man) in the user Maildir folder that you want to setup a filter:

| /home/vpopmail/domains/domain.com/user/filter
/usr/home/vpopmail/domains/domain/user/Maildir/

Save it and take care with the permissions, vpopmail user should be able to read it. What we are doing here is really simple, in the first line we pipe incoming emails trough filter, and according to filter exit code qmail execute (or not) the second line and proceed with the normal delivery.

Now setup the filter itself, it’s written in dirty and messy PHP but it gets the job done. Also it depends on PEAR Mail_mimeDecode, so go ahed and install it:

pear install Mail_mimeDecode

Now the filter script itself, it must be customized to your needs:

#! /usr/local/bin/php
<?php

/*
 * QMAIL FILTER EMAILS
 *
 * invoked by .qmail files
 * | /path/to/this/script
 *
 * parses email to address and subject against target strings
 * if BOTH are matched email is saved in $save_matched_dir
 * and qmail is instructed to ignore further .qmail lines
 *
 * CONFIG:
 */

$max_bytes        = 262144;            // mail size > 256Kb is not scanned
$to_address       = 'to@domain.com';   // to address filter
$subjects         = array('subject 1',
                          'match this subject',
                          'other subject');
$save_matched_dir = '/home/vpopmail/domains/domain.com/save-matched-emails/';

/*
 * QMAIL EXIT CODES
 *
 * 0 - Success (go to next .qmail line)
 * 99 - Success and abort (do not execute next lines)
 * 100 - permanent error (bounce)
 * 111 - soft error (retry later)
 */

try {
  function decodeHeader($input) {
    // Remove white space between encoded-words
    $input = preg_replace('/(=\?[^?]+\?(q|b)\?[^?]*\?=)(\s)+=\?/i', '\1=?',
                         $input);

    // For each encoded-word...
    while (preg_match('/(=\?([^?]+)\?(q|b)\?([^?]*)\?=)/i', $input, $matches)) {

      $encoded  = $matches[1];
      $charset  = $matches[2];
      $encoding = $matches[3];
      $text     = $matches[4];
      
      switch (strtolower($encoding)) {
        case 'b':
          $text = base64_decode($text);
          break;

        case 'q':
          $text = str_replace('_', ' ', $text);
          preg_match_all('/=([a-f0-9]{2})/i', $text, $matches);
          foreach($matches[1] as $value)
            $text = str_replace('='.$value, chr(hexdec($value)), $text);
          break;
      }

      $input = str_replace($encoded, $text, $input);
    }

    if (! isset($charset))
      $charset = 'ASCII';

    $input = strtolower(
               preg_replace('/[^a-z ]/i', 
                            '', 
                 iconv($charset, 
                       'ASCII//TRANSLIT//IGNORE', 
                       $input)
               )
             );
    return $input;
  }

  $mail  = '';
  $bytes = 0;

  $fr = fopen("php://stdin", "r");
  while (!feof($fr)) {
    $mail .= fread($fr, 1024);
    $bytes += 1024;
    if ($bytes > $max_bytes) {
      fclose($fr);
      exit(0);
    }
  }
  fclose($fr);

  require_once 'Mail/mimeDecode.php';
  $decoder   = new Mail_mimeDecode($mail);
  $structure = $decoder->decode(array('decode_headers' => true));

  // check from address
  $patt = '/[a-z0-9]+([_\\.-][a-z0-9]+)*@([a-z0-9]+([\.-][a-z0-9]+)*)+\\.[a-z]{2,}/i';
  preg_match($patt, 
             $structure->headers['to'], $matches);
  if (isset($matches[0]) && $matches[0] == $to_address) {
    // check subject
    $structure->headers['subject'] = decodeHeader($structure->headers['subject']);

    foreach ($subjects as $subject) {
       if (strpos($structure->headers['subject'], $subject) !== false) {
         $fw = fopen($save_matched_dir.
                     time().
                     '.'.
                     rand(1000, 99999).
                     '.'.
                     gethostname().
                     'S='.
                     strlen($mail).
                     ':2', 'w');
         fwrite($fw, $mail);
         fclose($fw);
         exit(99);
         // exit(0);
       }
    }
  }
} catch (Exception $e) {
}


// default, continue normal processing
exit(0);

?>

save, add the php hash bang, mark it executable and vpopmail owned. Also, the $save_matched_dir is not created and should already be present in your system.

Thats it. After this setup you should start seeing a steady flow of matched emails being saved in the matched directory and not delivered in your Inbox. As usually this works like a charm to me but can work incredible bad for you, so use at your own peril.

Car care – AC clean and spare tyre stow

The AC clean is a very important procedure to get a good air quality inside a vehicle, it can avoid nasty bugs like Legionella developing . It is a very easy procedure, any DIYer should be able to do it with good results and saving some good money (really dont understand how some shops charge over 50 euros for this…). So, what you need:

  • AC cleaner product – i use the 1z klima-cleaner due to very good reviews online – you can get it at a paint shop or car parts shop .
  • Philips screwdriver
  • New cabin filter with charcoal element (if outside big cities get the simple no need of the charcoal).

So first, gain access to the cabin filter and remove it. The Golf 4 is very easy, lift rain rubber seal, remove the screws that hold the filter cover (located in engine bay in front of passenger seat), press clips of the filter frame and pull out. With the old filter out of the way, clean everything up (dust, leaves, cigar butts…).


© Volksbloggin

Now comes the very hard part (not), shake the AC cleaner can and insert the hose that comes with it all the way down the AC system thru the openings revealed by the filter remotion. Use half of the can there, the other half use evenly in the air vents. Remember, AC off and air blower off, open the vents and select in the dash the corresponding air direction as you deliver the product in each vents/section. After the can is empty, put the new filter in place (installation is the reverse of the removal). Then, open the windows and let AC pump air for 10 minutes with nobody inside. You can now enjoy the fresh air inside your car, so go buy something pretty to yourself with the money you have just saved.

I also cleaned up the mess that was the spare tyre compartment. I had a flat (a big rupture at tyre wall) and needed to change the tyre at the highway shoulder, so the wrecked tyre had lots of debris attached to it, brake dust, road dust, bits of tyre rubber, etc that were laid up to the spare tyre compartment. Everything normal, i just don’t understand why when afterward you get the car to the tyre shop they couldn’t care less about that mess, and don’t even pass up with a rag… what a pigs… so unless you want to carry at the trunk brake dust, bits of burned rubber in decomposition and other things that are not good to your health, its up to you to clean up, its very easy and need simple tools:

  • Portable vacuum cleaner
  • Water
  • Nivea lotion

Remove the spare tyre, wash it up with water. Let it dry. Vacuum all the shit of the tyre compartment and clean up with a rag. When the tyre has dried apply a coating of Nivea lotion to the rubber, better more than less. Check and inflate if needed. Now you can store the spare tyre and rest assured that will serve you good in case of need.