Homemade NAS Box Redux

In 2012 I took my old PowerMac G4, slapped in a SATA card, some hard drives, and turned it into a server for storing media and computer backups. Then in 2013 I upgraded my MacBookPro to OS X 10.9 and lost the ability to do Time Machine backups to AFP shares from OS X 10.5. What to do, what to do? Solution: turn the PowerMac into a true server by running Linux. The results? Totally awesome.

The Hardware:

Full details at http://lookanotherblog.com/?p=323

  • 2001 Qucksilver PowerMac G4
  • FirmTek SeriTek/1V4 SATA PCI-X controller
  • 2x 2TB Western Digital Caviar Green 3.5” hard drives (3Gb/s)
  • 1x 320GB Western Digital Scorpio 2.5” hard drive
  • 1x 120GB PATA hard drive (more on this later)

The OS:

I did a lot of research on what operating system to pick for the PowerMac. PPC was been officially dead in Apples eyes back in 2007 with the WWDC announcement of OSX moving to Intel. Coupled with the PS3 loosing “Other OS” boot capability in 2010 [link], many Linux distros dropped official PPC support years ago. The only officially support non-OS X operating system I could find for PPC was FreeBSD, which I had played with in the past when looking at building a FreeNAS box.

Ultimately I decided to settle on Ubuntu 14.04 Server Edition for my OS. Canonical officially dropped PPC support in Ubuntu back in 2007 with the release of 7.04 [link], but the community has been maintaining unofficial builds of every release. The binary repository for Ubuntu PPC is large and mostly up to date, which makes installing software much easier than compiling from source. Also, there is a Java runtime package available for Ubuntu PPC, which makes running some applications I use actually possible.

The OS Installation Saga:

So I picked my variant of Linux and burned the ISO to a DVD. The next few steps should had been pretty straightforward.

  1. Boot computer from DVD
  2. Install OS onto 320GB SATA hard drive
    • turn on SSH and SAMBA
  3. Reboot computer
  4. See computer sitting at login prompt

Only, there was an issue. After installing Ubuntu on the hard drive and rebooting yaboot couldn’t find the OS. I did a week worth of research and different versions of Ubuntu, FreeBSD, and other non-OSX operating systems, but none of them could boot from my SATA drive. Finally I discovered a solution to the problem: Install the OS on the master PATA drive. I don’t know if this was a limitation of yaboot or PPC’s OpenFirmware, but the only way I got Linux to boot was by making sure it was installed on the very first hard drive. Crisis averted, on to the next set of steps. This same rule applied to G5 PowerMacs and their SATA drives in testing. Linux had to be installed on the top drive.

Network Shares:

To get network sharing back I had to completely redo my hard drive formatting with EXT3/4 and setup Samba/Netatalk. At least with these packages I didn’t have to do any crazy GPEdit or PLIST editing to get network backups to work. NOTE: there is a bug in the PPC build of FDISK that causes it not to properly report the sector size of a hard drive. If you need to partition a hard drive use DDISK instead.

I installed Samba during the initial OS installation so all that was left was setting up the config file.

security = user
[homes]
comment = Home Directories
browseable = no
read only = no
create mask = 0755
directory mask = 0755

[Data1]
comment = Archive Location
path = /shares/Data1
browsable = yes
guest ok = no
read only = no

[Media]
comment = Media Location
path = /shares/Media
browsable = yes
guest ok = no
read only = no

[TM_backup]
path = /shares/Data1/TM_backup
browseable = yes
read only = no
guest ok = no
comment = System Backups

After that I made sure Samba was configured to run on startup and restarted the service

sudo update-rc.d samba defaults
sudo service samba restart

The package of Netatalk in the Ubuntu repositories wasn’t up to date, only being at v2.X. This meant that I had to build/install Netatalk from source to get all the latest features and bug fixes (especially for Time Machine backups). Overall the process was pretty easy. Netatalk had a wiki page detailing the exact steps needed to build the software.

http://netatalk.sourceforge.net/wiki/index.php/Install_Netatalk_3.1.6_on_Ubuntu_14.04_Trusty

sudo apt-get install avahi-daemon build-essential libssl-dev libgcrypt11-dev \
   libkrb5-dev libpam0g-dev libwrap0-dev libdb-dev libmysqlclient-dev libavahi-client-dev \
   libacl1-dev libldap2-dev libcrack2-dev systemtap-sdt-dev libdbus-1-dev libdbus-glib-1-dev \
   libglib2.0-dev tracker libtracker-sparql-0.16-dev libtracker-miner-0.16-dev

./configure \
--with-init-style=debian-sysv \
--with-cracklib \
--enable-krbV-uam \
--with-pam-confdir=/etc/pam.d \
--with-dbus-sysconf-dir=/etc/dbus-1/system.d \
--with-tracker-pkgconfig-version=0.16
make
sudo make install

The configuration file for Netatalk was just simple as the Samba one.

nano /usr/local/etc/afpd.conf

.


;
; Netatalk 3.x configuration file
;

[Global]
; Global server settings

[Homes]
basedir regex = /home

[Data1]
path = /shares/Data1

[Media]
path = /shares/Media

[TM_backup]
path = /shares/Data1/TM_backup
time machine = yes

All that was left was to start the service and configure it to run on startup.

sudo service avahi-daemon start
sudo service netatalk start
sudo update-rc.d avahi-daemon defaults
sudo update-rc.d netatalk defaults

With all of the network sharing setup it was time to move on re-installing the applications I used.

Media Streaming/Playback

Over the past 3 years the playback needs I had morphed and changed. I caved and signed up for cable TV server from Insight (now Time Warner) so I could keep up with F1, IndyCar, Grand-AM/Unites SportsCar, and other forms of racing. Instead of getting the cable company provided set-top box I chose to rent a cableCARD and purchase a TiVo instead. Way back in 2005 I purchased my mother a TiVo Series 2 to replace the VCR she had and had fun using its dual tuners. Since that time TiVo released their Series3/HD and Series4/Premiere boxes, both which supported CableCARD.That was a long, roundabout way of saying I started using my PS3 less for streaming playback and wanted to use the TiVo instead.

Streambaby

http://sourceforge.net/projects/streambaby/.

The first application I got installed on the server was Streambaby. This application allowed me to stream (not transfer) mp4 and mp2 videos to the Tivo with minimal effort. If the formatting was correct then Streambaby didn’t have to transcode the video. A developer recently started developing the application again and added new features like playing mp2/mp4 mkvs without transcoding and more. Setup was simple only requiring the Jave runtime engine and FFMPEG to be installed.

As I said earlier, there is a package of Java’s JRE for PPC Linux from Ubuntu which is why I chose that distribution. FFMPEG, however, required being compile from source. Originally I tried using the deb-multimedia repository for FFMPEG, but I found many package conflicts when trying to install other media packages, such as Handbrake.

Install FFMPEG

https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu

Installed dependencies:

sudo apt-get -y --force-yes install autoconf automake build-essential libass-dev \
   libfreetype6-dev libgpac-dev libtheora-dev libtool libvorbis-dev pkg-config texi2html zlib1g-dev
mkdir ~/ffmpeg_sources

Compiled libx264-dev

cd ~/ffmpeg_sources
wget http://download.videolan.org/pub/x264/snapshots/last_x264.tar.bz2
tar xjvf last_x264.tar.bz2
cd x264-snapshot*
./configure --enable-static
make
sudo make install

Compiled fdk-aac

sudo apt-get install unzip
cd ~/ffmpeg_sources
wget -O fdk-aac.zip https://github.com/mstorsjo/fdk-aac/zipball/master
unzip fdk-aac.zip
cd mstorsjo-fdk-aac*
autoreconf -fiv
./configure --disable-shared
make
sudo make install

Compiled lamemp3

sudo apt-get install nasm
cd ~/ffmpeg_sources
wget http://downloads.sourceforge.net/project/lame/lame/3.99/lame-3.99.5.tar.gz
tar xzvf lame-3.99.5.tar.gz
cd lame-3.99.5
./configure --prefix="$HOME/ffmpeg_build" --enable-nasm --disable-shared
make
sudo make install

Compiled FFMPEG

cd ~/ffmpeg_sources
wget http://ffmpeg.org/releases/ffmpeg-snapshot.tar.bz2
tar xjvf ffmpeg-snapshot.tar.bz2
cd ffmpeg
./configure \
  --pkg-config-flags="--static" \
  --enable-gpl \
  --enable-libass \
  --enable-libfdk-aac \
  --enable-libfreetype \
  --enable-libmp3lame \
  --enable-libtheora \
  --enable-libvorbis \
  --enable-libx264 \
  --enable-nonfree
make
sudo make install
Install Streambaby

Installed Java JRE

sudo apt-get install wget unzip openjdk-7-jre-headless

Downloaded Streambaby from http://sourceforge.net/projects/streambaby/files/

Moved app to a folder under /opt

sudo unzip streambaby*.zip -d /opt/
sudo mv /opt/streambaby* /opt/streambaby

Configured the Streambay config file

sudo nano /opt/streambaby/streambaby.ini

.


#
# ./streambaby --help for more configuration file options
#

dir.1=/shares/Media/Movies
dir.1.name=Movies
dir.2=/shares/Media/TV/
dir.2.name=Top Gear
dir.3=/shares/Media/Streams
dir.3.name=Streams

Created a service daemon and register it

sudo nano /etc/init.d/streambaby

#!/bin/bash
### BEGIN INIT INFO
# Provides: streambaby
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# required-start:
# required-stop:
### END INIT INFO

DAEMON_PATH="/opt/streambaby"

DAEMON=streambaby
DAEMONOPTS=""

NAME=streambaby
DESC="My daemon description"
PIDFILE=/var/run/$NAME.pid
SCRIPTNAME=/etc/init.d/$NAME

case "$1" in
start)
printf "%-50s" "Starting $NAME..."
cd $DAEMON_PATH
PID=`java -Djava.net.preferIPv4Stack=true -Xmx256m -jar ./jbin/$DAEMON.jar $DAEMONOPTS > /var/log/%NAME.log 2>&1 & echo $!`
#echo "Saving PID" $PID " to " $PIDFILE
if [ -z $PID ]; then
printf "%s\n" "Fail"
else
echo $PID > $PIDFILE
printf "%s\n" "Ok"
fi
;;
status)
printf "%-50s" "Checking $NAME..."
if [ -f $PIDFILE ]; then
PID=`cat $PIDFILE`
if [ -z "`ps axf | grep ${PID} | grep -v grep`" ]; then
printf "%s\n" "Process dead but pidfile exists"
else
echo "Running"
fi
else
printf "%s\n" "Service not running"
fi
;;
stop)
printf "%-50s" "Stopping $NAME"
PID=`cat $PIDFILE`
cd $DAEMON_PATH
if [ -f $PIDFILE ]; then
kill -HUP $PID
printf "%s\n" "Ok"
rm -f $PIDFILE
else
printf "%s\n" "pidfile not found"
fi
;;

restart)
$0 stop
$0 start
;;

*)
echo "Usage: $0 {status|start|stop|restart}"
exit 1
esac

.

sudo chmod +x /etc/init.d/streambaby
sudo update-rc.d streambaby defaults

Transmission BitTorrent Client

Azureus was a good BitTorrent client for OS X a few years ago, but it got re-branded, added features that didn’t I need, dropped PPC support, and felt slow and bloated on old hardware. For the PPC Linux build I needed a client that was lightweight, but customizable. That basically left two options: rTorrent and Tranmission. I looked closely at both applications and test both for a few days and ultimately went with Transmission for the following reasons: web gui, supported by flexget as an output, easy to use. Don’t get me wrong, I got rTorrent running and doing everything I wanted with the highly customizable config file. I just didn’t enjoy using the cli interface.

Install Transmission
sudo apt-get install transmission-cli transmission-daemon

Reconfigure Transmissions settings

sudo service stop transmission-daemon
sudo adduser transmission-daemon sharing

.

sudo nano /etc/transmission-daemon/settings.json
{
    "alt-speed-down": 50,
    "alt-speed-enabled": false,
    "alt-speed-time-begin": 540,
    "alt-speed-time-day": 127,
    "alt-speed-time-enabled": false,
    "alt-speed-time-end": 1020,
    "alt-speed-up": 50,
    "bind-address-ipv4": "0.0.0.0",
    "bind-address-ipv6": "::",
    "blocklist-enabled": false,
    "blocklist-url": "http://www.example.com/blocklist",
    "cache-size-mb": 4,
    "dht-enabled": true,
    "download-dir": "/shares/Media/Transmission",
    "download-limit": 100,
    "download-limit-enabled": 0,
    "download-queue-enabled": true,
    "download-queue-size": 5,
    "encryption": 1,
    "idle-seeding-limit": 30,
    "idle-seeding-limit-enabled": false,
    "incomplete-dir": "/home/debian-transmission/Downloads",
    "incomplete-dir-enabled": false,
    "lpd-enabled": false,
    "max-peers-global": 200,
    "message-level": 2,
    "peer-congestion-algorithm": "",
    "peer-id-ttl-hours": 6,
    "peer-limit-global": 200,
    "peer-limit-per-torrent": 50,
    "peer-port": 51413,
    "peer-port-random-high": 65535,
    "peer-port-random-low": 49152,
    "peer-port-random-on-start": false,
    "peer-socket-tos": "default",
    "pex-enabled": true,
    "port-forwarding-enabled": false,
    "preallocation": 1,
    "prefetch-enabled": 1,
    "queue-stalled-enabled": true,
    "queue-stalled-minutes": 30,
    "ratio-limit": 2,
    "ratio-limit-enabled": true,
    "rename-partial-files": true,
    "rpc-authentication-required": false, #Yes, I know I shouldn't do this.
    "rpc-bind-address": "0.0.0.0",
    "rpc-enabled": true,
    "rpc-password": "{c0d7e4133acba8e3a02d2428d044ef2277138ae9YXpmBleN",
    "rpc-port": 9091,
    "rpc-url": "/transmission/",
    "rpc-username": "transmission",
    "rpc-whitelist": "127.0.0.1",
    "rpc-whitelist-enabled": false,
    "scrape-paused-torrents-enabled": true,
    "script-torrent-done-enabled": false,
    "script-torrent-done-filename": "",
    "seed-queue-enabled": false,
    "seed-queue-size": 10,
    "speed-limit-down": 768,
    "speed-limit-down-enabled": true,
    "speed-limit-up": 256,
    "speed-limit-up-enabled": true,
    "start-added-torrents": true,
    "trash-original-torrent-files": false,
    "umask": 18,
    "upload-limit": 100,
    "upload-limit-enabled": 0,
    "upload-slots-per-torrent": 14,
    "utp-enabled": true
}

Restart Transmission

sudo service transmission-daemon start
sudo update-rc.d transmission-daemon defaults

Install FlexGet

I figured out a better setup for FlexGet and properly set it up as a service on the system
http://flexget.com/wiki/InstallWizard/Linux/Environment

sudo apt-get install python-pip
sudo pip install flexget
sudo pip install transmissionrpc
sudo adduser --system --shell /bin/false --no-create-home --group flexget
sudo adduser flexget sharing
sudo adduser flexget syslog
sudo mkdir /usr/loca/etc/flexget
sudo chown flexget:flexget /usr/local/etc/flexget
sudo pip install --upgrade six>=1.70 #only required for Ubuntu 14.04

Create a config file. Right now I am using FlexGet to download podcasts.

sudo nano /usr/local/etc/flexget/config.yml
Tasks:
  TekThing:
    inputs:
      - rss: http://feeds.feedburner.com/Tekthing
    accept_all: yes
    download: /shares/Media/Streams/TekThing
  TWIT:
    inputs:
      - rss: http://feeds.twit.tv/brickhouse_video_hd.xml
    series:
      - Before You Buy
      - Know How
    download: /shares/Media/Streams/{{series_name}}
  cleanup:
    clean_transmission:
      host: localhost
      port: 9091
      transmission_seed_limits: yes

Create the FlexGet Daemon.

sudo nano /etc/default/flexget

# Configuration for /etc/init.d/flexget

# User to run flexget as.
# Daemon will not start if left empty.
FGUSER="flexget"

# Full path to the flexget config.yml file to use.
# Defaults to FGUSER $HOME/.flexget/config.yml
CONFIG="/usr/local/etc/flexget/config.yml"

# Path to the directory where flexget should log. Do not add trailing slash.
# Defaults to the FGUSER $HOME/.flexget directory
LOG="/var/log"

# Log verbosity
# Available options : none critical error warning info verbose debug trace
# Defaults to info
LEVEL="info"

And create the FlexGet daemon script

sudo nano /etc/init.d/flexget

#!/bin/bash

### BEGIN INIT INFO
# Provides:          flexget
# Required-Start:    $network $remote_fs
# Required-Stop:     $network $remote_fs
# Should-Start:      
# Should-Stop:       
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Flexget
# Description:       FlexGet is a multipurpose automation tool 
#                    for content like torrents, nzbs, podcasts,
#                    comics, series, movies, etc.
### END INIT INFO

# Author: Antoine Joubert, 19/01/2014

NAME="flexget"
DAEMON="/usr/local/bin/flexget"
SETTINGS="/etc/default/$NAME"

DESC="Flexget"
PIDFILE="/var/run/$NAME.pid"

set -e

. /lib/lsb/init-functions

unset FGUSER CONFIG LOG LEVEL

# Exit if flexget not installed
if [ ! -x "$DAEMON" ]; then
        log_action_msg "$DESC: Could not find flexget executable. Exiting."
        exit 2
fi

# Read configuration variables
if [ -r /etc/default/$NAME ]; then
        . /etc/default/$NAME
else
        log_action_msg "$DESC: /etc/default/$NAME not found. Exiting."
        exit 2
fi

# Exit if FGUSER has not been set in /etc/default/flexget
if [ -z $FGUSER ]; then
        log_action_msg "$DESC: FGUSER not set in /etc/default/$NAME. Exiting."
        exit 2
fi

# Function to verify if flexget is already running
run_check() {
        if [ -e $PIDFILE ]; then
               status_of_proc -p $PIDFILE $DAEMON $NAME > /dev/null && RETVAL=0 || RETVAL="$?"
        else
                RETVAL="2"
        fi
}

end_log() {
        if [ $RETVAL -eq 0 ]; then
                log_end_msg 0
                return 0
        else
                log_end_msg 1
                exit 1
        fi
}

# Function to define config file, log file and log level
conf_check() {
        if [ -z $CONFIG ]; then
                OPTIONS="$OPTIONS"
        else
                OPTIONS="-c $CONFIG"
        fi
        if [ -z $LOG ]; then
                OPTIONS="$OPTIONS"
        else
                OPTIONS="$OPTIONS -l $LOG/flexget.log"
                if [ ! -d $LOG ]; then 
                        mkdir -p -m 750 $LOG
                        chown $FGUSER $LOG
                fi
        fi

        if [ -z $LEVEL ]; then
                OPTIONS="$OPTIONS"
        else
                OPTIONS="$OPTIONS -L $LEVEL"
        fi
}

start_flexget() {
        run_check
        if [ $RETVAL = 0 ]; then
                log_action_msg "$DESC: Already running with PID $(cat $PIDFILE). Aborting."
                exit 2
        else
                conf_check
                log_daemon_msg "$DESC: Starting the daemon."
                start-stop-daemon --start --background --quiet --pidfile $PIDFILE --make-pidfile --chuid $FGUSER \
                --user $FGUSER --exec $DAEMON -- $OPTIONS daemon start
                RETVAL=$?
                end_log
        fi
}

stop_flexget() {
        run_check
        if [ $RETVAL = 0 ]; then
                log_daemon_msg "$DESC: Stopping the daemon."
                start-stop-daemon --stop --quiet --chuid "$FGUSER" --pidfile "$PIDFILE" --retry 30
                RETVAL=$?
                [ -e "$PIDFILE" ] && rm -f "$PIDFILE"
                end_log
        else
                log_action_msg "$DESC: Not currently running. Aborting."
                exit 2
        fi
}

status_flexget() {
        run_check
        if [ $RETVAL = 0 ]; then
                log_action_msg "$DESC: Currently running with PID $(cat $PIDFILE)."
        else
                log_action_msg "$DESC: Not currently running."
        fi
        exit $RETVAL
}

case "$1" in
        start)
                start_flexget
        ;;
        stop)
                stop_flexget
        ;;
        restart)
                stop_flexget && sleep 2 && start_flexget
        ;;
        status)
                status_flexget
        ;;
        *)
                echo "Usage: $0 {start|stop|restart|status}"
esac

exit 0 

And make the script executable

sudo chmod +x /etc/init.d/flexget
sudo update-rc.d flexget defaults

Conclusion

And that is mostly it. All the functionality I had under OS X is recreated using Ubuntu. I’ve continued to add functionality to the server, which I hope to expand upon in other posts.

Until then, have fun.