Docker on RaspberryPi

Installing Docker on a RaspberryPi might seems overkill, but it actually works pretty well, even if installing the tools can be a little messier if you don’t know how to do it and you start searching the Internet for clues.

After being intimidated by the many alternatives given to you to overcome the lack of a docker package on the official package repositories you’ll start wondering which might be a secure way to start playing with Docker without compromising your system security: here is what I have done.

The Client and the Server

First thing you need is to have the Docker server along with the command line client installed.

The RaspberryPi Foundation, within their blog, recommend to simply use:

$> curl -sSL https://get.docker.com | sh

That’s an official shell script which will identify your operating system and architecture (don’t forget we are on an ARM processor) and install the most appropriate binaries.

The Compose

After playing for a while with Docker you’ll start to feel the urgency to use the docker-compose tool, which sadly is another missing element.

I’ve found the simplest way to have it running is by executing the following two commands:

$> sudo apt-get -y install python-pip
$> sudo pip install docker-compose

Advertisements

nRF24 walk through – Building the sensors

It’s now time to physically build the sensor devices part of the nRF24 sensor network we are planning to build, so move to your electronics bench and pick some breadboards!

From an electronics point of view each sensor node consists of:

• 1 Arduino (any low-end model will do, I will be using two nano and one pro micro as this is what I have at hand)
• 1 push button
• 1 nRF24L01P module
• 1 1-100uF capacitor rated for 5V or more (usually, but not necessarily, electrolytic – any value in the range will do)
• 1 voltage source for 3.3V capable to deliver at least 20mA; viable options include:
  • the 3V3 pin of your Arduino board (available only on my two nano)
  • a voltage divider made by two resistors (I will use 100Ω and 220Ω allowing 50mA to pass through)

WARNING While the module I/O pins are 5V tolerant (which makes this module compatible with practically every microcontroller), the module requires a 3.3V power source.

For absolutely no reason you should power the nRF24L01 module from a 5V source, unless you want to permanently damage your radio transceiver!

Please note I’m compromising here to keep things simple as the 3.3V source is definitely non optimal if you want to save on power consumption: for battery powered sensors you should be using 3.3V Arduinos, like the 8MHz pro mini, or an efficient voltage regulator like the AMS1117-3.3.

To keep the bill of materials at it’s bare minimum I’ll use the Arduino internal pull-up feature: this eliminates one resistor, but implies that pin 3 (the one attached to the button along with GND) will be HIGH when not pressed and turns LOW when pressed, just keep this in mind when we’ll start developing the sensors’ firmware.

NOTE These modules are quite sensitive to power line fluctuations and it should not be surprising considering they are radio transceivers operating in the 2.4GHz frequency: when transmitting data they require short but intense energy bursts to generate the electromagnetic field carrying the data. The capacitor we are adding should be as close as possible to the module in order to provide a local source of energy and avoid power fluctuations: it’s not an optional component!

I will build 3 of those using, more or less, the following wiring:

The breadboard view for a Pro Mini	The associated schematics for a Pro Mini
Breadboard view for a Nano	The associated schematics for a Nano

If you wonder where I found the nRF24L01 Fritzing part, the answer is I made one myself (you can get it here).

Believe it or not, this is how these drawings turned into reality on my desk:

I’m going to build three adapters to plug the nRF24 modules onto my breadboards.	I’ve used a few bits of wire to ensure connection on these single side protoboards
The components on a half size breadboard, wiring still missing	The nRF24 module plugged into its socket: isn’t it a beauty?
And here they are in all their splendor! Two are Arduino Nano clones, the rightmost one is an Arduino Pro Micro clone.

Now, the important thing to remember when wiring the modules, is they use a Serial Peripheral Interface, also known as SPI. What that means? Well, whatever board you are using to communicate to your modules, you need to identify the pins associated with the MISO, MOSI and SCK functions. Luckily for us their location is pretty well documented for Arduino and Arduino-like boards.

The modules use two additional I/O pins, called CE and CSN, but they are configurable so you can pick your own: I went for pins 9 and 10 respectively. Bear in mind though, if you’ll ever decide to change those pins, you will have to change the software accordingly.

NOTE Most microcontrollers have one single SPI interface, this doesn’t mean you can have only one SPI device attached though! Most SPI devices, including nRF24, have a CSN (Chip Select Not) pin which instructs the device to temporarily ignore SPI communication, thus allowing for communicating with other SPI devices.

Testing

Now that you have your sensor boards ready, let’s move onto the software part.

You might want to use the Arduino IDE or, as I will, a more advanced solution like the Eclipse Arduino Plugin, an open source project I’m proud to contribute whenever I get the opportunity to. In case you decide to go for the Eclipse based solution, please follow the website instructions as not every configuration is supported.

I will use a few open source libraries trying to reduce the code to the real interesting parts:

• the RF24 from TMRh20 library is a fork of the original library from Maniacbug, who is no longer active
• my MicroDebug library, which I’ll use to provide some development feedback on serial console

Before getting your hands dirty though, it’s preferable to check your wiring with a known working firmware.

Among the many examples available in the RF24 library, the one I will use is RF24/examples/pingpair_ack.ino: this Arduino sketch is pretty simple and provides enough info to verify your setup.

There are just a couple of changes I’m going to make, to adapt to the above wiring, and I’ve marked them with comments in this Gist:

1. the pins used as CE and CSN are pins 9 and 10, so line 24 has been changed from RF24 radio(7,8) to RF24 radio(9,10) in order to reflect that
2. there’s no need to run the serial connection at 57600bps, so I slowed it down to 9600bps in line 41
3. one of my boards uses the ATmega 32u4 chip (Leonardo and Micro use it), which means a board reset does not reset the serial connection; I need to wait for serial connection in the setup() function to be able to check the module config, this required the addition of line 43

Once the changes are in place, upload the sketch onto the first sensor board and connect using the serial terminal. Don’t worry if you don’t see anything printed out, the addition of line 43 requires you to send something to unlock the setup phase.

As soon as you send a character, you should see the following appear onto your screen:

An example of invalid setup.

The serial output generated by a successful initialization of the nRF24 control library

As you can see the radio.printDetails() function call at line 59 provides tons of information: most will be blank (0xff) in case your module has not been setup correctly (99.9% as a consequence of bad wiring or false connections). If everything is setup correctly, you should see the addresses used at line 27 printed in the serial console.

Once you have verified the first module, leave it powered and move on to another one, but this time we will go a step further verifying the two modules are actually talking each other: if the second module reports a correct configuration send the character T on the serial and watch the two talking each other.

IMPORTANT When moving to verify the next board you must ensure to remove power to one of the other two, because this sketch uses and forces a linear network topology of two elements only: if you try to power up another one you will be unable to distinguish who is replying to whom.

In the next article we will look at the sensors’ firmware and how we can wireless notify the hub about the sensor’s button state change.

nRF24 walk through – Sensors’ firmware

Let’s move to the 3rd part of the nRF24 walk through series before the excitement cools off, shall we?

Now that I have three boards wired and I’m certain only my code can break things up, it is time to create the sensors’ firmware.

It should be quite simple: all we need to do is getting the transceiver set up at the beginning and send the current sensor identifier every time the button is pressed.

As this series is intended to be a tutorial, I’ll expand a little on the basic requirement. To add a bit of salt on our sensors’ network communication, we will expect the hub to send back to the nodes 2 pieces of information in response to their click events:

• the total count of clicks received from all nodes is going to be part of this article and we’ll achieve that using the ack packet payload
• the number of clicks received from the specific node will be instead added in another post, so not to push too much info in one shot

Each device is going to require a unique identifier: as radio network address (think ofit like an IP address) and for the software to print out (like a computer name). I will use one byte stored in EEPROM for both: this will limit the maximum amount of sensors for this network to 255 (0 is going to be reserved as the hub identifier/address).

That’s not such a low number after all, but we want our hub to print out the node identifier as a letter, so I will further limit the range to [1, 26] to simplify the hub code.

NOTE The above limits to the number of nodes is very soft and has been introduced into this tutorial to simplify the code. The real hard limit to the number of unique nodes is much higher and in the order of 1 thousand billions (1 followed by 12 zeros)!

Setting things up

The radio transceiver configuration occurs, as you might expect, in the setup() function, plus a global variable declaration and a few defines:

// Creates a new instance using SPI plus pins 9 and 10
RF24 radio(9, 10);

// nRF24 address family: all addresses will be in the format 0xFACEC0DE## with the last two
// bytes determined by the node identifier.
#define ADDR_FAMILY 0xFACEC0DE00LL
#define MAX_ID_VALUE 26

#define BUTTON_PIN 3

// This node unique identifier: 0 is used for the hub, anything above MAX_ID_VALUE is considered
// not valid
byte nodeId = 255;

void setup() {
  SERIAL_DEBUG_SETUP(57600);

  // Setup the push button
  pinMode(BUTTON_PIN, INPUT_PULLUP);

  // Read the address from EEPROM
  byte reading = EEPROM.read(EEPROM_ADDR);

  // If it is in a valid range for node addresses, it is our address
  if (reading > 0 && reading <= MAX_ID_VALUE) {
    nodeId = reading;
    DEBUG("Node identifier is %c", nodeId + 64);

    // Initialize the transceiver
    radio.begin();
    radio.setAutoAck(true); // Enables auto ack: this is true by default, but better to be explicit
    radio.enableAckPayload(); // Enables payload in ack packets
    radio.setRetries(1, 15); // Sets 15 retries, each every 0.5ms, useful with ack payload
    radio.setPayloadSize(2); // Sets the payload size to 2 bytes
    radio.setDataRate(RF24_2MBPS); // Sets fastest data rate, useful with ack payload

    // Opens the first input pipe on this node address
    radio.openReadingPipe(1, ADDR_FAMILY + nodeId);

    // Opens the first input pipe on this node address
    radio.openReadingPipe(2, ADDR_FAMILY + 254);
#if (SERIAL_DEBUG)
    // Prints current configuration on serial
    radio.printDetails();
#endif
  } else {
    DEBUG("Invalid node id %u found: use S## (a number between 1 and %u) to configure the board", reading, MAX_ID_VALUE);
  }
}

There are a few details in the code related to the nRF24 module which might be valuable to explain:

• when I instantiate the module driver class I’m specifying only two out of the five pins used by module (not including the power line) because the MISO, MOSI and SCK pins are defined by the board SPI interface
• I’m defining a network address class 40 bits long (0xFACEC0DE00), not very different from IP addressing when you specify the network class (192.168.0.0 is the network class C of an host having IP address 192.168.0.*)
• I’m enabling auto acknowledge packets (useful to know if your transmission was successful) capable to carry payloads (more on this later)
• I’m instructing the library to auto retry packet transmission every half millisecond (500 us) up to 15 times, so to reduce transmission errors
• I’m setting a fixed payload size, which improves transmission reliability, of 2 bytes
• the data rate is set to highest possible value (2Mb per second), so to have the ack packets coming back in time for being recognized (more on this in the next article)
• one of the input pipes (the second if we count in pipe 0) is set to this node address. This is not used to receive ack packets (pipe 0 is used for that), but it will come into play later on, when I’ll show how the sensors can receive data (other than ack packets) from the hub (or whoever wants to talk to us, for what it matters!)

IMPORTANT Reading pipes are not associated to addresses you talk to, but to addresses you want to listen for. If that sounds confusing, consider reading pipes like mailboxes: you can have multiple mailboxes, but that does not limit who can send you mail, actually anybody knowing at least one of them can.

This means you are not limited to receive data from maximum five nodes (as commonly and mistakenly known) and this walk through will demonstrate it!

NOTE If you want to know more about pipes and addressing, the best source of information is the datasheet, but to summarize:

• you have one writing pipe only, but you can change it’s value to whatever address you want to talk to
• there are six reading pipes, only two (0 and 1) can be associated with 40 bits addresses, with pipe 0 automatically associated to the writing address (to receive acks); this means pipes 1 to 5 share the same upper 32 bits, but you can change the reading pipes addresses
• almost every address is valid, only a few are excluded (like 0x0000000000, 0xFFFFFFFFFF and similar)
• address length is configurable to 40 (default), 32 or 24 bits, but the limitation on pipes 1 to 5 still applies as they keep sharing the upper 24 or 16 bits

For the sensors’ setup() function to complete correctly we need to read a valid node id from EEPROM which means we have to push something in the correct location: this is responsibility of another function called config(). I’m not going to analyze that function in detail, but it should be clear its purpose is to allow to set the node id using a serial connection.

A looping life

Moving into the sensor loop() function, the plan is even easier, if possible: check the button state and, if pressed, send the sensor node identifier to the hub.

To add value to this tutorial we will expect the hub to send back the total amount of received clicks and we will use ack packet as the carrier of this information.

NOTE Acknowledge (here abbreviated into ack) packets are used to verify data transmission has been successful. This is a common technique used in TCP/IP protocol as well: if you don’t feel comfortable with the concept just consider those as return receipts.

The protocol used by this chip allows to optionally attach some information to the return receipts: we refer to this info as the ack packet payload.

Ack packets maximum payload size depends on data rate and re-transmission speed and is explained in details in the nRF24 datasheet at paragraph 7.5.2.

I’m going to use some macros to perform a simple software debouncing on the push button, but apart from that, the code should be very easy to read:

// Button debouncing macros
#define DEBOUNCE 15
#define DMASK ((uint16_t)(1<<DEBOUNCE)-1)
#define DF (1<<(uint16_t)(DEBOUNCE-1))

// Macro for detection of falling edge and debouncing
#define DFE(signal, state) ((state=((state<<1)|(signal&1))&DMASK)==DF)

// Button debouncing status store
unsigned int buttonStatus;

void loop() {
  // Checks if we are trying to configure the node identifier
  config();

  delay(1); // Slow down a bit the MCU

  // Checks the push button: if we enter here the button has been pressed
  if (DFE(digitalRead(BUTTON_PIN), buttonStatus)) {

    // Sets the destination address for transmitted packets to the hub address
    radio.openWritingPipe(ADDR_FAMILY);

    // Put transceiver in transmit mode
    radio.stopListening();

    // Transmit this node id to the hub
    bool write = radio.write(&nodeId, 1);

    DEBUG("Send attempt from node %c was %s", nodeId + 64, write ? "successful" : "UNSUCCESSFUL");

    // Get acknowledge packet payload from the hub
    while (write && radio.available()) {
      unsigned int count;
      radio.read(&count, 2);

      // This function shows how a node can receive data from the hub
      // without using ack packets payload
      receiveNodeCount();

      DEBUG("Got response from hub: total click count is %u", count);
    }
  }
}

Once again, let’s analyze in detail the parts specific to the nRF24 module, which are activated whenever the push button state has gone LOW for enough time:

• I set the transmission destination address using the openWritingPipe() function, providing the hub node identifier (that is the node we want to send data to)
• to put the module into transmit mode we need to call the stopListening() function
• since ack packets have been enabled, we can transmit reliably, so the result of the write() function (a boolean value) is going to be stored and checked
• the data I’m going to transmit (the sensor node identifier, stored into the nodeId global variable) is provided to the write() function along with the number of bytes (only one) we are going to send. The actual payload sent on the air is going to be 2 bytes long, because we have set the packet size to a fixed value of 2, which means the packet will be zero filled
• if the transmission was successful we expect to have an ack packet available. We have enabled ack packets payload, so we can pull out an unsigned int (2 bytes) into the count variable: this represents the total amount of clicks the hub has received so far

This is a complete example of data exchange, but the fact we are using the ack packet payload has some caveats we will analyze when we will talk about the hub.

The whole picture

The complete sensor’s firmware is available on Github and its compilation requires the two libraries mentioned in the previous post: the RF24 lib from TMRh20 and my own MicroDebug lib.

Please ignore the receiveNodeCount() function call and its implementation for now: we’ll get into that in another post. If you prefer, comment out line 105 to completely exclude that part of the firmware.

Naming the newborns

To avoid packet transmission conflicts we will have to initialize each sensor board with its own unique identifier: that’s the part handled into the config() function.

Internally I will use numeric identifiers, but they will be translated into one character when printing, hence the limit to numerical values between 1 and 26: that’s the number of letters in the latin charset.

Each board will initially have 255 as identifier, which is to be considered invalid, but you can change it via serial.

Once connected, type the S character followed by the board identifier you want to assign (like S2 or s5): the board will print the corresponding identifier as a character, store it in EEPROM and self reconfigure.

IMPORTANT The communication relies entirely on assigned node identifiers. The hub has no other way than the node identifier to distinguish between the nodes.

If you configure two nodes with the same identifier they will be considered as one and their clicks will not be distinguished. You can leverage this in some situations and have two sensors appear as one, if that’s what you want.

Sensor node id configuration is very simple and can be done via serial console in a handful of seconds

Repeat the procedure on each board and you have your sensors setup!

From this point forward your sensors does not require to be hooked up to your computer, they will just need power. If you have enough USB ports available and decide to leave the sensors connected to your computer though, you will be able to receive some debug messages on the serial ports and enjoy the occurring communication… once we have the hub running!

The Arduino Eclipse Plugin supports multiple consoles and distinguishes among them using different colors: black text is from the hub, red, blue and green from the sensors!

nRF24 walk through – Introduction

The Nordic nRF24 is a family of silicon integrated radio transceivers operating in the 2.4GHz band, the most popular one being the nRF24L01. This is the core element of some extremely cheap module boards available in online stores like eBay, Aliexpress and Banggood.

These boards do not provide WiFi (801.11) or Bluetooth connectivity (both in the 2.5GHz band), but they can be used to establish custom wireless networks between small electronic devices, including Arduino, RaspberryPi and Particle (formerly known as Spark).

Whenever we talk about networks you must take in account a few key aspects of networking, one of the most important being the network topology.

During this series we will aim to establish a star network between a series of Arduino based peripheral nodes and a central hub node, being either a RaspberryPi or a Particle Core/Photon: this represents a basic but invaluable configuration allowing for complex elaborations on remotely collected information, either on premise (RPi) or in the cloud (Particle).

If this series gets enough attention I might invest some more time and extend it to cover more complex topologies like tree and mesh, with the latter being my favorite and, IMHO, most valuable for inexpensive IoT projects.

The project

To keep things simple our peripheral nodes will be only collecting button presses, communicating to the central hub whenever a button gets pressed: the central hub will periodically (once every 30 seconds) print out the amount of button clicks it has received with a breakdown for each node; something like:

Received 14 clicks in the past 1 minute(s)
* 5 click(s) from node A
* 2 click(s) from node B
* 7 click(s) from node F

This will obviously represent just an example of what you will be able to do from the hub node; nothing prevents you, as an example, from pushing data into a database and generating graphs. You could aggregate the data differently or, more likely, collect other types of data from your sensor nodes: I’m not here to place constraints to your imagination!

Keep in mind though, the little radio transceivers we are using have a few limitations that are commonly misunderstood, which will be analyzed when we get there during the project.

The steps

This walk through will be split into the following posts:

• Building the sensors
• Sensors’ firmware
• Network details
• Arduino hub
• Particle hub
• RaspberryPI hub

XBian from USB with SD fallback

After the big mess I’ve previously posted about I’ve decided to get the opportunity to switch over to a better and more performing setup for my Raspberry Pi, not to mention the addition of multiple backups…

The goal I set up was to have mi Pi running off a root filesystem on my external USB hard drive while still being able to be portable, so running off the SD when needed.

I took the opportunity to switch from RaspBMC to XBian, mostly because the latter is based on Raspbian which seems to have a better support on the forums.

I started by flashing an XBian RC2 image onto my SD card from a Windows laptop using the tool provided by the community (the Linux version didn’t work for me).

After being surprised by the fact this distro doesn’t use ext3/4 but btrfs (I didn’t even know it existed) and it already includes the kernel module for my WiFi dongle I started messing with the boot sequence: I wanted to be able to drop to command line before the init sequence started and manipulate the cmdline.txt to change the rootfs location.

Because my keyboard seems to use a weird, non standard, firmware, I suddenly discover I was unable to use it to drop into rescue mode so I had to revert to a somewhat automated way to switch to SD card root fs.

I ended up with a little more complicated solution which performs fully automated fallback: if the USB disk isn’t connected at boot time then it reverts to SD card and reboots. Once the USB disk gets connected back though, you need to execute a command to restore the USB root.

To achieve this I had to create a little shell script:

#!/bin/sh

fallback() {
	$mount_bin /dev/mmcblk0p1 /mnt && echo "Boot filesystem mounted"
	if [ ! -f /mnt/cmdline.usb.txt ] && [ -f /mnt/cmdline.sd.txt ]; then
		echo "Switching back to SD card root filesystem..."
		mv /mnt/cmdline.txt /mnt/cmdline.usb.txt
		cp /mnt/cmdline.sd.txt /mnt/cmdline.txt
		umount /mnt
		exit 0
	else
		umount /mnt
		echo "Fallback procedure unavailable"
		exit 1
	fi 
}

restore() {
	if [ -f /boot/cmdline.usb.txt ]; then
		echo "Re-enabling USB root filesystem..."
		mv -f /boot/cmdline.usb.txt /boot/cmdline.txt
	else
		echo "Nothing to do"
	fi
	exit 0
}

case $1 in
	fallback) 
		fallback;;
	restore) 
		restore;;
	*)
		echo "Usage: `basename $0` ( fallback | restore )";;
esac

As you can see this script provides two functions: you can either try to fallback from USB to SD or to restore your boot sequence to use a USB root filesystem.
You can obviously use this from command line, something you will have to if the fallback procedure occurs, so I created a symbolic link to it into /usr/sbin to have it on my superuser execution path.
Because I want this file to be available at boot time I need to have it within the initramfs, that’s why I created this file as /etc/xbian-initramfs/usb-boot.sh.
But this is not enough as the initramfs image is created by a script which doesn’t include the file unless you explicitly tell it to: to do so I edited the update-initramfs.sh script adding the following highlighted lines to include my new creation:

cp /etc/xbian-initramfs/cnvres-code.sh ./
cp /etc/xbian-initramfs/splash_updater.sh ./
# Add root fs USB to SD fallback/restore script
cp /etc/xbian-initramfs/usb-rootfs.sh ./

copy_with_libs /usr/bin/stdbuf
copy_with_libs /usr/lib/coreutils/libstdbuf.so
copy_with_libs /usr/bin/setterm

Now you need to have a copy of the /boot/cmdline.txt file which runs nicely from SD card and name it /boot/cmdline.sd.txt: I performed the copy at this stage as so far everything was working fine for me.

At this stage you might want to move your root filesystem over your USB hard drive and this is going to be the easiest task: you can do it through the xbian-config command by simply choosing option number 6 and specifying the partition on your USB hard disk. With XBian RC2 this does practically everything for you!

Next step is to enable the initramfs at boot as it is disabled by default to save a little on boot time: edit /boot/config.txt uncommenting the initramfs line.

Before creating the new initramfs image let’s tweak the init process a little to automate the fallback procedure: remember, my keyboard is not recognized during the boot sequence so I will not be able to hold down the shift key…

The file to alter is /etc/xbian-initramfs and the following highlighted lines are the contributions which automate the fallback check and reboot if needed:

if [ "$CONFIG_rootfstype" != "nfs" ]; then
    while ! get_root && [ -n "$CONFIG_rootwait"  -a  "$X" -lt "$CONFIG_rootwait" ]; do
	/sbin/modprobe -q usb_storage
	X=$(($X+1))
	echo "$X Waiting for root..."
        sleep 1
    done
    if [ ! -b "${CONFIG_root}" ]; then
	echo "Root partition ${CONFIG_root} missing"
	eval ./usb-rootfs.sh fallback
	if [ "$?" -eq '0' ]; then
	    echo "Fallback root filesystem applied: rebooting now!"
	    sync
	    reboot -f -d2
	fi
	drop_shell
    fi
    ln -s ${CONFIG_root} /dev/root
    up "root block device found"

    export FSCHECK=$(blkid -s TYPE -o value -p ${CONFIG_root})
    resize_part; resize_ext4; convert_btrfs
    up "after resize, convert"
fi

Practically the added section steps into the root partition identification and right after the declaration of a missing root partition tries to perform a fallback procedure: if it succeeds it reboots the system, otherwise the process continues as normal and the user is dropped into the rescue shell.

Customize Transmission on Fonera 2.0N

I finally managed to nicely setup my Fonera 2.0N torrent client to work as I expect, even if it was not a very simple task.

What I wanted was to use a separate in-progress folder for non completed torrents and a completed folder for… guess what!

I found there are two ways to achieve this goal: having SSH access to the Fonera or having a Linux distro (a live one will do).

Preparation

In both cases you need to plug into the Fonera an USB 2.0 hard drive. I recommend not to use flash drives as they are much slower and will die quite quickly, not considering their capacity is a lot smaller: I used a 250GB USB 2.0 Maxtor hard drive I had lying around.

You don’t have to format the hard drive if it’s formatted FAT32 or ext2/ext3, but both the FOnera team and I recommend against using an NTFS formatted hard drive as it will slow down everything. Remember though that FAT32 has a maximum file size limit (biggest file it can store) of 4GB which can be easily hit if you are used to download Blue Ray images or any other big file format: I went for an ext3 file system which can accommodate all my needs and I will use Paragon ExtFS in case I wish to plug this hard drive into a Windows computer.

Once plugged the hard drive will get assigned a generated name (something like Disk-A1) which I didin’t like as it doesn’t tell much about the functionality and can get confusing if you use multiple hard drives:  I went into the USB Disk section and assigned it the name TORRENT (all uppercase) as this is going to be the disk only purpose.

Whatever is the method you will use, you need to initialize your hard drive for running the torrent client (Transmission is its name) by setting it up into the Torrent section: please ensure the drive name listed here corresponds to the name you assigned to the drive in the USB Disk section as we will use it.

After setting up the disk to run the torrent client (yes, the torrent client binaries and configuration is going to be hosted onto the external hard drive) start it and wait for the process to complete.

Customize through SSH

I start with this method as I think it’s the easiest one if you have flashed a DEV firmware which enables SSH access.

Get access to your Fonera using root as username and your Fonera WPA key as password (the default one is printed on the side of your modem), then get into the Transmission startup script located at /tmp/images/torrent/bin/start.sh.

In this file you’ll find a very long line containing startup instruction for the transmission-daemon client, and by default it contains two directives that are going to override our next customization. We will get rid of them both by removing the part --download-dir $1/torrent -c $1/torrent.

Save the file and let’s switch to a web browser to configure our transmission client through the Transmission web UI: now you will be able to change the Download folder that was previously forced to be torrent by the script we just changed: now it can be anything you like, but the folder must exists on the disk, so create it if it’s not already there. I decided to use a folder called completed, in opposition to in-progress which will store the non completed torrents. I kept the torrent folder in case I wish to upload .torrent files into the disk instead of using the web UI (probably it will never be used).

Shut down the Transmission client: this operation will write into a file your configuration.

Now we need to access the Transmission configuration without having the Transmission client running. When you start the torrent client your Fonera will open three files hosted on your hard drive within the FoneraApps folder and mount them as disks. We will do the same, but we’ll mount only one of them, the torrent.2.3.7.1.var file (the numbering might be slightly different and depends on your firmware version).

Switch back to the SSH console and issue the following commands to create a mount point and mount the disk image into it:

mkdir /tmp/torrent.var
mount -o loop /tmp/mounts/TORRENT/FoneraApps/torrent.2.3.7.1.var /tmp/torrent.var

Now you can edit /tmp/torrent.var/settings.json changing the download-dir, incompleted-dir and incompleted-dir-enable options to your desired folders, the download-dir one should already look correct as it was set by the Transmission web UI.

Mine look like:

"download-dir": "/tmp/mounts/TORRENT//completed",
"incomplete-dir": "/tmp/mounts/TORRENT//in-progress",
"incomplete-dir-enable": "true",

Now ensure those folders exist or make them yourself:

mkdir /tmp/mounts/TORRENT/completed
mkdir /tmp/mounts/TORRENT/in-progress

Customize on Linux

If you don’t have SSH access to the Fonera you can still modify the Transmission configuration as it is completely stored onto your hard drive: just shutdown the torrent client on the Fonera through the web UI and move your hard disk to your Linux box.

Once mounted you’ll find a couple of new folders that have been created by the Fonera: FoneraApps and torrent. Inside the former you’ll find three files:  torrent.2.3.7.1.fmg, torrent.2.3.7.1.var and torrent.2.3.7.1.swp (the numbering might be slightly different as it depends on your Fonera firmware version).

Let’s start with the fmg file, which I guess it stands for Fonera Image, by mounting it through the following:

cd <your usb disk mount point>
mkdir /tmp/torrent.img
sudo mount -o loop FoneraApps/torrent.2.3.7.1.fmg /tmp/torrent.img

We will have to modify the content of the /tmp/torrent.img/bin/start.sh file containing startup instruction for the transmission-daemon client: by default it contains two directives that are going to override our next customization. We will get rid of them both by removing the part --download-dir $1/torrent -c $1/torrent.

Save the file and let’s move to the next step: unmount this disk image and mount the var one with:

sudo umount /tmp/torrent.img
sudo mount -o loop FoneraApps/torrent.2.3.7.1.var /tmp/torrent.img

Now you can edit /tmp/torrent.img/settings.json changing the download-dir, incompleted-dir and incompleted-dir-enable options to your desired folders, mine look like:

"download-dir": "/tmp/mounts/TORRENT//completed",
"incomplete-dir": "/tmp/mounts/TORRENT//in-progress",
"incomplete-dir-enable": "true",

Now save the file, unmount the image and ensure those folders exist:

umount /tmp/torrent.img
rmdir /tmp/torrent.img
cd <your usb disk mount point>
mkdir completed
mkdir in-progress

Unmount your usb drive, unplug from your Linux box, plug it back into the Fonera and restart the Transmission client to enjoy your new custom configuration!

Recover your (very) deleted files

Yesterday I did something stupid. Ok, that’s not the very first time it happens to me, but I believe this specific time it was something worth sharing.

My RaspberryPi did run out of space and all the files I was sharing through the attached 2TB USB hard drive were unreachable. I took the opportunity to perform a “system upgrade” I had planned for a while: run the entire system off an USB hard drive, another 350GB I had laing around.

So I hooked up the new drive and issued what I knew was an unsafe command, but without using the necessary caution:

dd if=dev/mmcblk0 of=/dev/sdc bs=4M

I realized sdc was my storage hard drive only when the process did end and I wasn’t able to access it locally… WTF!

I was speechless, my face became angry and I started looking at the wall in search of a nice spot where some red splash would have looked nice.

I started searching for backups, but I knew I had recently performed a nice cleanup recently in order to re-organize my archives.

After two hours spent in vain searching through my 4 USB hard drives I was so upset with myself I moved my storage hardrive onto Windows and formatted it NTFS.

I knew there are some tools to scan your hardrive and guess the sector contents, but I knew they would have taken ages to scan a 2TB drive attached via USB and what I would have got at the end would have probably been a bunch of sector backups with no significance…

Then I stumbled upon this super easy and super smart piece of free software called Recuva: with a few clicks and with no need for a degree in hard drive construction I was able to recover a huge amount of my files (well, I didn’t even try to recover those that got even just a sector overwritten by my smart command) and it took about 12 hours from installation to a directory full of files.

And all this after a sector by sector overwrite (dd) and a Windows fast format!!

Well, it’s not what I call an happy ending, but it’s better than nothing and it’s definitely worth sharing: don’t be stupid like me!

nRF24 on Raspberry Pi

NOTE This post has been quite successful, so I decided to publish a complete series on nRF24 transceivers.

I’m working on a home automation project and I’m planning to use my Raspberry Pi as central node of a network of cheap nRF24 nodes.

First thing is to get the necessary compilation tools, something quite easy to achieve even on my RaspBMC installation:

$> apt-get install build-essential

With the compiler and the other tools at your hand you might want to get a library to get access to the nRF24 hardware, which is not a difficult step to achieve either:

$> git clone https://github.com/stanleyseow/RF24.git
$> cd RF24
$> cd librf24-rpi/librf24
// compile the files
$> make
// install the library
$> sudo make install
// check the library availability
$> sudo ldconfig -v | grep librf
      librf24.so.1 -> librf24.so.1.0

Now, let’s move to the wiring, but DO NOT attempt any connection while your Raspberry is powered up: even a brief short circuit made with a floating cable getting contact for a fraction of millisecond can ruin your day.

I’ve found some small difficulties here, mainly due to misleading information. Please refer to the pictures below and click on them to enlarge.

Once you have everything in place you can power up your Pi and starting to get some fun!

All the code below is available as a Gist on GitHub for your convenience.

You can start with the examples contained within the library itself, but if you feel brave enough here is the code I’m executing on my Pi:

// file payload.h
#ifndef __VIRIDI_NEXU_PAYLOAD__
#define __VIRIDI_NEXU_PAYLOAD__

enum PayloadType {
    HERBA, METEO
};

typedef uint8_t vn_payload_type;
typedef uint8_t vn_payload_version;

typedef struct {
        int16_t humidity;
        int16_t temperature;
        int16_t pressure;
        int16_t altitude;
        int16_t luminosity;
} vn_meteo_t;

typedef struct {
        int16_t moisture;
        int16_t temperature;
} vn_plant_t;

struct Payload {
        vn_payload_type type;
        vn_payload_version version;

        union {
            vn_meteo_t meteo;
            vn_plant_t plant;
        } data;
};

#endif // __VIRIDI_NEXU_PAYLOAD__

// file receiver.cpp
#include <cstdlib>
#include <iostream>
#include "librf24/RF24.h"
#include "payload.h"
using namespace std;

//
// Hardware configuration
//
RF24 radio("/dev/spidev0.0", 8000000 , 25);  //spi device, speed and CSN,only CSN is NEEDED in RPI
const uint64_t pipes[2] = { 0xF0F0F0F0E1LL, 0xF0F0F0F0D2LL };

Payload payload = Payload();

//
// Setup
//
void setup(void) {
  radio.begin();
  radio.setRetries(15, 15);
  radio.setDataRate(RF24_250KBPS);
  radio.setPALevel(RF24_PA_MAX);
  radio.setPayloadSize(sizeof(payload));
  radio.openWritingPipe(pipes[1]);
  radio.openReadingPipe(1, pipes[0]);
  radio.startListening();
  radio.printDetails();
}

//
// Loop
//
void loop(void) {
  if (radio.available()) {
    radio.read(&amp;amp;amp;amp;amp;payload, sizeof(payload));
    printf("packet %d %d %d %d %d \n", payload.data.meteo.temperature, payload.data.meteo.humidity, payload.data.meteo.luminosity, payload.data.meteo.altitude, payload.data.meteo.pressure);
  }
}

int main(int argc, char** argv){
        setup();
        while(1)
                loop();

        return 0;
}

To build the above code you just save the two files and execute gcc providing the necessary parameters: if you get an error regarding a missing header file you might have to adjust the librf/RF24.h include directive accordingly to your build location.

$> g++ -Wall -Ofast -mfpu=vfp -mfloat-abi=hard -march=armv6zk -mtune=arm1176jzf-s -L../librf24/ -lrf24 receiver.cpp -o receiver

Remeber you must be have access to the /dev/spidev device to execute your program, the simplest way to get such permission is by running your receiver code as root:

$> sudo ./receiver

Now let me give you one last advice: always take in consideration differences in hardware architecture when you program/compile your communcation software!
In particular, if you send something from an Arduino (8 bit microcontroller) take in consideration there’s an important difference when the same data is read from the Raspberry (32 bit ARM processor):

• float numbers are 4 bytes long on Arduino and 8 bytes on Raspberry
• enumeration are 2 bytes on Arduino and 4 bytes on Raspberry
• any struct on Raspberry is padded to even bytes
• and so forth

What this means while developing for the nRF24 chips is that you need to take in consideration the architecture differences when transferring data and ensure the data types can be handled by both ends of the communication channel: my suggestion is to avoid floating points and stick to integers. If you need to transfer fractional values, switch the number to a higher scale: for example instead of transferring a temperature sensor reading of 25.12 C°  (Centigrade degrees), multiply the value by 1000 and transfer it as 25120 mC° (milli Centigrade degrees).

On the Arduino side there are many tutorials out there, but for the sake of completeness here is my wiring and the simple software sending out the payload for the Raspberry to read.

And here is the Arduino sketch broadcasting the data into the air

#include "payload.h"

#define SERIAL_DEBUG true
#include <SerialDebug.h>

#define LED_DEBUG true
#include <LedDebug.h>

#define SENSE_DELAY 2000
Payload payload = (Payload) { METEO };

#include <RF24Network.h>
#include <RF24.h>
#include <SPI.h>

uint32_t lastSense;
inline bool sense() {
	long now = millis();
	if (now < lastSense || now - lastSense > SENSE_DELAY) {
		payload.data.meteo.humidity = millis();
		payload.data.meteo.temperature = millis() * 50;
		payload.data.meteo.pressure = millis();
		payload.data.meteo.altitude = millis();
		payload.data.meteo.luminosity = map(analogRead(A0), 0, 1024, 100, 0);
		lastSense = now;
		return true;
	} else {
		return false;
	}
}

#define RADIO_CE_PIN 9
#define RADIO_CS_PIN 10
#define RADIO_RETRY_DELAY 15
#define RADIO_RETRY_COUNT 15
RF24 radio = RF24(RADIO_CE_PIN, RADIO_CS_PIN);
const uint64_t pipes[2] = { 0xF0F0F0F0E1LL, 0xF0F0F0F0D2LL };

void setup() {
	SERIAL_DEBUG_SETUP(9600);
	pinMode(A0, INPUT);

	radio.begin();
	radio.setRetries(RADIO_RETRY_DELAY, RADIO_RETRY_COUNT);
	radio.setDataRate(RF24_250KBPS);
	radio.setPALevel(RF24_PA_MAX);
	radio.setPayloadSize(sizeof(payload));
	radio.openWritingPipe(pipes[0]);
	radio.openReadingPipe(1,pipes[1]);
}

void loop() {
	if (sense()) {
		radio.powerUp();
		if (!radio.write(&payload, sizeof(payload))) {
			PULSE(3,75);
		} else {
			PULSE(1,225);
		}
		radio.powerDown();

		DEBUG("payload", sizeof(payload), "enum", sizeof(payload.type));
	}
}

 

Artifactory and NamedVirtualHost

Today I needed to publish an Artifactory repository I’ve set up for my company to a first level URL, something like http://artifactory.mycompany.com: it revealed to be not as straightforward as I supposed to.

Once haveing installed Tomcat, deployed the Artifactory WAR and checked that everything was running fine I needed to map the http://artifactory.mycompany.com URL to the Tomcat webapp so I opted for the standard Apache HTTPD VirtualHost section and the common mod_proxy_ajp ProxyPass and ProxyPassReverse directives, but it was not working as expected.

Whenever I was requesting the initial URL I was ending up having http://artifactory.mycompany.com/artifactory/webapp in my web browser address bar and a Tomcat error shown in there referring to a non available resource at http://artifactory.mycompany.com/artifactory/artifactory/webapp (please note the additional artifactory folder).

Apparently Artifactory performs some sort of magic to distinguish the client type and redirects to an appropriate view, but in doing this it forcibly prefix each redirection with the webapp name, invalidating my configuration.

I ended up with the following Apache HTTPD configuration which seems working, but it’s probably sub optimal as I’m not an HTTPD expert:

  ServerName artifactory.mycompany.com
  ServerAlias artifactory.mycompany.com

  ProxyPreserveHost on

  ProxyPass /artifactory ajp://localhost:8009/artifactory
  ProxyPassReverse /artifactory ajp://localhost:8009/artifactory
  ProxyPass / ajp://localhost:8009/artifactory/
  ProxyPassReverse / ajp://localhost:8009/artifactory/

Mercurial & Git Client

I just discovered there’s a nice tool developed from Atlassian allowing client access to Git and Mercurial repositories with a nice integration with both Bitbucket and Github, two free source hosting platforms I frequently use.

The tool name is SourceTree, it’s available for free for Windows and MacOS and it looks damn good and complete: it took a while to download all the components on a fresh environment but it seems solving the SCM client problem in a sleek way!