Raspberry PI running from an external HDD

This is a scenario where you have an external HDD (or “pen” drive) connected via USB to the PI. The SD card will be just used for the boot sequence then everything will work from the external HDD, I find the system to work faster, smoother and more stable this way (at least with my v1 PI).

First thing is to normally install the Raspbian system in the SD card. There will be 2 partitions on it. A first small fat16 partition labeled ‘boot’ with the lba flag set, and a second ext4 partition. So, on your external HDD you will need also a ext4 partition. I advice you to to make 2 partitions in the HDD, one for the OS, programs and files, and the second (like a /home/ or /data/) for personal files, backups, etc. In the event of a major catastrophic malfunction, this configuration offers a bigger safeguard to your personal data.

If you need assistance to partition and format your HDD probably its a good idea to stop here.

Mount and sync the second SD card ext4 partition with your external HDD ext4 partition

sudo rsync -avHx /mnt/sdcard/ /mnt/hdd

Next step is to mount the SD card boot partition. And change in /cmdline.txt (adjust sda1 to your HDD partition as needed).

root=/dev/mmcblk0p2 

to

root=/dev/sda1 rootwait

You can actually delete de SD card system directory (or not) it works the same. Also don’t forget to add an entry in /etc/fstab for mounting the second partition in the HDD in case you have it.

Good luck with your PI.

Procmail with Qmail + Vpopmail

Following the qmail threads in this blog, and after a successful experience filtering emails in the server with a php script! time was upon to thinker a bit with the elder of all email server filters, Procmail – the mail processing utility for Unix.

Whe are talking really old stuff, as Wikipedia states the initial release in 1990, so 26 years from this writing, and about a zillion years in computer time (a date so old that it’s closer to the Unix Epoch than it is of today).

As usual, the easy part in FreeBSD is the installation:

cd /usr/ports/mail/procmail
make install

there. Now the tricky part, that is to make it play nicely with Qmail+Vpopmail setup. For the first experiences you probably should setup a couple of test accounts.

The concept is pretty simple, for an account that you want to filter email with Procmail we are going to add/or change the .qmail file that controls the email delivery to a filter script that invokes procmail and throw back a proper qmail exit code according to Procmail result.

It is very important to take into account that in this setup Procmail DOES NOT deliver the email directly, it filters the email, and according with the recipes rules, it can stop the delivery chain, forward the email, invoke an external command, etc.

Without further delays. The customized .qmail that calls the filter script:

| /home/vpopmail/domains/mydomain.com/teste/procmail_filter
/usr/home/vpopmail/domains/mydomain.com/teste/Maildir/

this is pretty simple and standard stuff. The qmail-local parses the .qmail file. The line with a pipe means to feed the message to the specified program. The command is invoked by qmail-command that runs sh -c command in the home directory, makes the email messsage available on standard input and setups some environment variables.

For this thread the most important stuff are the exit codes:

Command’s exit codes are interpreted as follows:
0 means that the delivery was successful;
99 means that the delivery was successful, but that qmail-local should ignore all further delivery instructions;
100 means that the delivery failed permanently (hard error);
111 means that the delivery failed but should be tried again in a little while (soft error).
Currently 64, 65, 70, 76, 77, 78, and 112 are considered hard errors, and all other codes are considered soft errors, but command should avoid relying on this.

With this info it’s pretty straight forward to devise the procmail_filter script.

#!/bin/sh

/var/qmail/bin/preline /usr/local/bin/procmail ./.procmailrc

status=$?

if [ $status -eq 99 ]
then
  status=99
elif [ $status -eq 0 ]
then
  status=0
elif [ $status -le 77 ]
then
  status=111
else
  status=100
fi

exit $status

and mark it executable.

The first line (after the shebang) the script calls qmail preline program, it simply inserts at the top of each message a UUCP-style From_ line, a Return-Path line, and a Delivered-To line because Procmail doesn’t understand the qmail environment variables. Calls procmail and sets Procmail configuration file the .procmailrc in the same directory.

Now for the .procmailrc stripped down to a very simple example:

SHELL = /bin/sh
LOGFILE=./pm.log
LOG="
"
VERBOSE=yes

# Exitcodes
# 0 normal delivery
# 99 silent discard a message
# 100 bounce a message

# Recipes

:0
* ^From: email@domain.com
{
  EXITCODE=99

  :0
  | /usr/local/libexec/dovecot/deliver -d delivertothis@email.com

}

# Avoid duplicates
:0
/dev/null

The top lines are the configuration variables, I would strongly suggest to use a log file for testing, in this example called pm.log that lives in the same directory. The strange LOG directive simple adds a new line to each log entry.

Then the recipes, in this example we match all emails from email@domain.com and deliver it to a local email delivertothis@email.com using Dovecot deliver command (so it takes care of Maildir quota), and set exitcode 99 to discard the message. Exit code 99 means that the delivery was successful, so all .qmail further delivery instructions will be ignored.

We could simply put an email address for a external email address, or even a local address but with less efficiency as it will trigger a new delivery process. This is auto-magical thanks to FreeBSD mailer.conf wrapper.

:0
* ^From: email@domain.com
{
  EXITCODE=99

  :0
  delivertothis@email.com

}

The last lines avoid duplicates, Procmail /dev/nulls the message and gives back the delivery control to the .qmail flow.

That’s it, everything playing nice with each other in old good UNIX tradition.

Customizing WordPress Jetpack Mobile Theme

JetPack by WordPressI have successfully using Jetpack Mobile Theme to serve the mobile version of my blogs, as of now it uses a theme called minieleven (that is starting to look a bit old…), with some tweaks here and there it can shine brighter and be customized to your taste, but unfortunately you can’t use a child theme, so other strategies must be used, so all the hard work isn’t wiped out by an update (and you do your updates, right?).

The plug-in native customization options are rather weak. You can customize, excerpts or full posts in home/archive/search pages, hide or show featured images, and display a promo for WordPress mobile apps. And that’s it, no more customization options for the mobile theme. Actually (never tried myself) if your main theme has colour and background options they should be picked up. And that’s it. From that point onward it’s up to you. So here we go…

First, if you are using Adsense for monetization, you will probably will notice that the Adsense mobile ads come with a nasty yellow background and cropped at the far right. The fix is very easy, go to Jetpack-> Options -> Custom CSS (be sure to check Mobile-compatible to Yes and simply add this:

.mobile-theme ins.adsbygoogle {
	background-color: transparent !important;
}

.mobile-theme ins.adsbygoogle iframe {
	width: inherit !important;
}

This is an important concept, as from here you can CSS override anything in the mobile theme, just be sure to prefix your selector with the .mobile-theme class, so you don’t mess up the desktop version.

This is all good for minor adjustments, and let’s say honestly this Jetpack feature is a little bit dumb, because it adds the custom CSS both to the mobile and the desktop version, actually to act on mobile you must switch an option and append the .mobile selector. This is dumb because to customize a desktop theme you simply create a child theme…. let’s say you want to use your own font with font-face, .mobile @font-face simply wont work.

The best bet for anything more than simple teaks is to create your own plug-in that pulls a specific style-sheet for the jetpack mobile version. A plug-in to customize other plug-in, wordpress is getting a bit tricky. happily this is quite easy, create a directory inside your /wp-content/plugins/ directory, let’s say jetpack-mobile-customizer and create a new file called jetpack-mobile-customizer.php with this content:

<?php
/**
 * @package Jetpack Mobile Customizer
 * @version 0.1
 */
/*
Plugin Name: Jetpack Mobile Customizer
*/

// Check if we are on mobile
function jetpackme_is_mobile() {

    // Are Jetpack Mobile functions available?
    if ( ! function_exists( 'jetpack_is_mobile' ) )
        return false;

    // Is Mobile theme showing/not showing?
    if ( isset( $_COOKIE['akm_mobile'] ) && $_COOKIE['akm_mobile'] == 'false' )
        return false;

    if ( isset( $_COOKIE['akm_mobile'] ) && $_COOKIE['akm_mobile'] == 'true' )
        return true;


    return true;
}

function jetpackme_add_css() {
    if ( jetpackme_is_mobile()) {
        wp_register_style('custom_jpmobile_style', 
                          '/wp-content/themes/my-child-theme/jpmobile_style.css');
        wp_enqueue_style('custom_jpmobile_style');
    }
}

add_action('wp_enqueue_scripts', 'jetpackme_add_css');

?>

and customize on line 26 your (child) theme and css. Now, just place all the CSS stuff on this file and it will be applied only to the jetpack mobile site.

To customize the number of posts showed on the mobile version (probably 5 posts it’s a bit overwhelming on hand-held devices), just add this code to your plug-in:

function custom_posts_per_page() {
    if ( jetpackme_is_mobile()) {
        return 1; // mobile version posts number
    } else {
        return false;
    }
}

add_filter('pre_option_posts_per_page', 'custom_posts_per_page', 10000);

And to customize the footer? Yes, again we go to the plug-in to add our stuff and hide the other stuff with CSS. On the plug-in add:

function jetpackme_custom_footer() {
    if ( jetpackme_is_mobile()) {
        global $wp;
        $current_url =  trailingslashit( home_url( add_query_arg( array(), $wp->request ) ) );
        echo '<span><a href="'.$current_url.
             '?ak_action=reject_mobile">Go to Desktop Version</a></span>'.
             '<p align="center">&copy; All rights reserved, yadayadayada</p>';
        return false;
    }
}

add_action('wp_mobile_theme_footer', 'jetpackme_custom_footer');

and in the CSS:

footer #site-generator {
        padding: 0 !important;
}

footer #site-generator > a {
        display:none;
}

So, there you go. Some clear ideas how to do the job of customizing the Jetpack Mobile Theme and surviving updates.

And now for a completely free bonus, a full customization of the menu and search. A more modern look and feel with everything hidden (menu items and search form) and of course replacing the ugly “Menu” and down arrow text with the ubiquitous 3 horizontal bars. Just open this blog with a mobile device or force the mobile view on this link.

We are using all the concepts previous explained and the very useful gettext filter. On the plug-in:

// Check if we are on mobile
function jetpackme_is_mobile() {

    // Are Jetpack Mobile functions available?
    if ( ! function_exists( 'jetpack_is_mobile' ) )
        return false;

    // Is Mobile theme showing?
    if ( isset( $_COOKIE['akm_mobile'] ) && $_COOKIE['akm_mobile'] == 'false' )
        return false;

    return jetpack_is_mobile();
}

function jetpackme_add_css_js() {
    if ( jetpackme_is_mobile()) {
        wp_register_style('custom_jpmobile_style', 
                          '/wp-content/themes/my-child-theme/jpmobile_style.css');
        wp_enqueue_style('custom_jpmobile_style');

        wp_register_script('custom_jpmobile_script', 
                           '/wp-content/themes/my-child-theme/jpmobile_script.js', 
                           array('jquery'));
    }
}

function fix_menu($translated_text, $text, $domain) {
    if ( jetpackme_is_mobile()) {
		switch ( $translated_text ) {
			case 'Menu' :
				$translated_text = '<span>Menu</span>';
				break;
			case 'Termo' :
				$translated_text = 'Pesquisar';
				break;
		 }
	}

	return $translated_text;
}

add_filter('gettext', 'fix_menu');
add_action('wp_enqueue_scripts', 'jetpackme_add_css_js');

The CSS:

.search-form #s, .menu-search {
	background-color: transparent !important;
}

.menu-search {
	-moz-box-shadow: none !important;
	-webkit-box-shadow: none !important;
	box-shadow: none !important;
	margin: 0 0.6em 0 !important;
	padding: 1% 2.5% !important;
	padding-right: 0 !important;
	padding-bottom: 0 !important;
	margin-bottom: 0 !important;
	width: auto !important;
	height: auto !important;
}

.menu-search::after {
	display:none;
}

#access {
	width:50% !important;
}

#access h3::before {
    content: "\f419";
    color: #666;
	font:1.3em "Genericons";
	line-height:46px;
}

#access h3.menu-toggle {
	color:#000 !important;
	padding: 0 !important;
	width: auto !important;
}

#access h3.menu-toggle span {
	display:inline-block;
	text-indent: -9999px;
}

#access .menu-toggle::after {
	display:none !important;
}

#access ul.nav-menu {
	left:0 !important;
	background-color:#f1f1f1 !important;
	-moz-box-shadow: none !important;
	-webkit-box-shadow: none !important;
	box-shadow: 0 2px 2px -2px #999 !important;
	padding: 0 !important;
	border-bottom:1px solid rgba(0,0,0,0.1) !important;
	margin-top:3px !important;
}

#access ul.nav-menu li {
	margin: 0 0.6em 0 !important;
	padding: 0 2.5% !important;
	border-bottom: 0 !important;
}

#access ul.nav-menu li a {
	display:block !important;
	padding:1em 0 !important;
}

#access ul.nav-menu::before {
	display:none !important;
}

.search-form {
	width:49% !important;
	display:none;
}

.search-form #s {
	border:1px solid #ddd !important;
	color:#999 !important;
	padding: 0.3em !important;
}

.search-form #s:focus {
	font-size: 1em !important;
	color:#000 !important;
	padding: 0.3em !important;
}

And the extra Javascript file loaded in the plug-in:

(function($) {
	$('h3.menu-toggle').click(function() {
		if ($(this).hasClass('toggled-on') == true) {
			$('div.search-form').show(500);
		} else {
			$('div.search-form').hide();
		}
	});
})(jQuery);

Comments, thoughts or other stuff feel free to use the comment box bellow.

CTT – Correios de Portugal check-digit

CTT - Correios de PortugalI was unable to find anywhere a Portuguese national postal service, package registry number check-digit validation. So, I politely asked them and soon after a PDF describing the algorithm was on my desk.

It’s pretty simple and standard stuff (only an awkward multiplier order of each digit), so the best way to describe it’s with a validation function in PHP (even for non PHP users/programmers should be simple to enough to understand at a glance):

function checkDigitCTT($ref) {
    if (! preg_match('/^[a-z]{2}[0-9]{9}[a-z]{2}$/i', $ref))
        return false;
            
    $digits      = substr($ref, 2, 9);
    $multipliers = array(8,6,4,2,3,5,9,7);
        
    $tmp = 0;
    foreach ($multipliers as $k => $v)
        $tmp = $tmp + (((int) $digits[$k]) * $v);
			
    $tmp = round($tmp % 11);

    if ($tmp == 0)
        $check_digit = 5;
    else if ($tmp == 1)
        $check_digit = 0;
    else
        $check_digit = 11 - $tmp;
			
    if ($check_digit == ((int) $digits[8]))
        return true;

    return false;
}

Raspberry PI open hotspot for your company site(s) only

Raspberry HotspotThe problem is really simple, you want/need to give open Wifi to your customers (let’s say inside a shop), but to you own company website (or websites) only. And nothing else, no other resources in the (internal or external) network.

The solution is simple and it comes in a tiny format… you will just need a Raspberry PI with a Wifi USB dongle that supports AP mode. Your company website should have an exclusive IP address

Side note: as normal, I’m not liable for any kind of mess, data loss, massive meteorite smash or other apocalyptic event in your world due to this guide.

Have the PI installed with the latest Raspbian, booted and logged in as root (sudo -s or equivalent).

Update the software sources:

apt-get update

Install the required software

apt-get install hostapd dnsmasq

Configure the wireless interface with a static IP address,
edit /etc/network/interfaces

iface wlan0 inet static
address 10.0.0.1
netmask 255.255.255.0
broadcast 255.0.0.0
# pre-up iptables-restore &lt; /etc/iptables.rules

and restart the interface

ifdown wlan0
ifup wlan0

Here I choosed the 10.0.0.1 address to isolate the Wifi guests from the 192.168.1.x internal network. You should adapt it according to your existing set-up.

edit /etc/default/hostapd

and replace
#DAEMON_CONF=””

with
DAEMON_CONF=”/etc/hostapd/hostapd.conf”

now edit (it’s a new file) /etc/hostapd/hostapd.conf

For a full list of switches and whistles please do refer to http://w1.fi/cgit/hostap/plain/hostapd/hostapd.conf, we go with a very minimalistic (but functional) configuration

interface=wlan0
ssid=WIFI-FREE-AS-BEER
channel=0
macaddr_acl=0
auth_algs=1
wmm_enabled=0
driver=nl80211
hw_mode=g
ieee80211n=1

Here we can start the service.

service hostapd start

and I got the dreadful failed in red font… a lsusb command quickly showed the infamous RTL8188CUS chip:
Bus 001 Device 004: ID 0bda:8176 Realtek Semiconductor Corp. RTL8188CUS 802.11n WLAN Adapter

Thanks to the good people of the Internets you get a quick fix (you are downloading an external binary… so cross your fingers before installation, and nothing bad will happen to your PI… well, it worked for me).

wget http://dl.dropbox.com/u/1663660/hostapd/hostapd
chmod 755 hostapd
mv /usr/sbin/hostapd /usr/sbin/hostapd.ori
mv hostapd /usr/sbin/

and change in /etc/hostapd/hostapd.conf
driver=nl80211
to
driver=rtl871xdrv

service hostapd start

service [….] Starting advanced IEEE 802.11 management: ok
hostapdioctl[RTL_IOCTL_HOSTAPD]: Invalid argument

Even with the warning output the service managed to start and work correctly.

By now there should be an open network called WIFI-FREE-AS-BEER available to log in, but the process will stall in the Obtaining IP Address stage. So it’s time to move to the DHCP and DNS server.

Edit /etc/dnsmasq.conf, and place at the end of the file the lines

address=/#/aaa.bbb.ccc.ddd
interface=wlan0
dhcp-range=10.0.0.10,10.0.0.250,12h015/05/raspberry-pi-open-hotspot-for-your-company-sites-only/
no-resolv
log-queries
log-dhcp

adjust the aaa.bbb.ccc.ddd to the exclusive public IP address of your company website. Basically we are configuring Dnsmasq to answer all name resolution queries to your public IP address, and setting DCHP leases to the Hostspot clients from IP 10.0.0.10 to 10.0.0.250 valid for 12h periods.

From now on it should be possible to log in to the Hotspot, but no data flow, so let’s take care of this now. First activate the kernel IP forwarding

echo 1 > /proc/sys/net/ipv4/ip_forward

and then adjust iptables rules

iptables -F
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
iptables -A FORWARD -i eth0 -o wlan0 -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -A FORWARD -i wlan0 -o eth0 -p tcp -d aaa.bbb.ccc.ddd --dport 80 -j ACCEPT
iptables -A FORWARD -i wlan0 -o eth0 -p tcp -d aaa.bbb.ccc.ddd --dport 443 -j ACCEPT
iptables -A FORWARD -i wlan0 -o eth0 -p udp -d 10.0.0.1 --dport 53 -j ACCEPT
iptables -A FORWARD -i wlan0 -o eth0 -p udp -d 10.0.0.1 --dport 67:68 -j ACCEPT
iptables -A FORWARD -i wlan0 -j DROP 

remember to replace aaa.bbb.ccc.ddd with your the exclusive public IP address like in dnsmasq. From this point there should be a fully functional system. You can login to the Hotspot, and any http/https request will be landing in your company website. All other network traffic (except for the DHCP and name resolution will be blocked).

Now, to wrap up just make all this stuff survive reboots:

echo "net.ipv4.ip_forward=1" >> /etc/sysctl.conf

update-rc.d hostapd defaults
update-rc.d dnsmasq defaults

iptables-save > /etc/iptables.rules

and uncomment in /etc/network/interfaces the line
# pre-up iptables-restore < /etc/iptables.rules

There is just one thing left, avoid the captive portal detection and the respective sign in to network message. If you are using some kind of URL mapping/decoupling system (really hope you do) it’s pretty easy.

For Android, test for http://clients3.google.com/generate_204 request and send a 204 header and 0 bytes:

if (isset($script_parts) && $script_parts[0] == 'generate_204') {
    header('HTTP/1.1 204 No Content');
    header('Content-Length: 0');
    die();
}

For iOS lalaland test for the user agent ‘CaptiveNetworkSupport’ and send a 200 response:

if (isset($_SERVER['HTTP_USER_AGENT']) && preg_match('/CaptiveNetworkSupport/i', $_SERVER['HTTP_USER_AGENT'])) {
    header("HTTP/1.1 200 OK");
    die();
}

That’s it folks, and I wonder what will be the next use for this tiny big computer?

UPDATE
After all been working well and good for a long time, maybe after a reboot a problem surfaced. Maybe a whim of the bits gods, the system was using the dnsmasq on internal lookups for all interfaces ignoring the interface directive.

So for example if one ssshed into the raspberry and tried to wget google.com one would get our company site…. not good.

Simple fix, manually edit /etc/resolv.conf, you can use Google public DNS (not censored) or your LAN Router IP (that normally uses the upstream DNS of your provider).

# Google IPv4 nameservers
nameserver 8.8.8.8
nameserver 8.8.4.4

and to not be automatic overwritten by dhcpclient updates set the immutable bit:

chattr +i /etc/resolv.conf

UPDATE 2
Noticed that the raspberry was missing /etc/network/interfaces (no file at all and I don’t recall to delete it). Maybe the problem was due to this and Maybe it’s time for a new SD card and fresh install.