Archive for August, 2010

I spent the last week writing this utility to manage my ZFS backups. ZFS’s snapshot ability is great for efficient incremental backups. Solutions using rsync and other file level backup utilities are at a disadvantage because they do not know what changes occur so they have to constantly scan all the data during each backup. Since ZFS’s incremental backups are made at the file system level, changes are marked during new write operations making the process of snapshot-ing instantaneous. I wrote this based on inspiration from duplicity and  ftplicity (which I think now is just duply?). The idea is to send remote snapshots to an offsite location securely and safely, that is, encrypt it so nobody can read your data and ensure that all your 1s and 0s don’t get mangled in the process. This utility tries to do both utilizing gnupg for encryption and md5sum for data integrity, though I maintain no guarantee this first version will actually meet up to both goals, I am pretty confident it will work in most circumstances. I still need further testing (yes, from you) for faulty conditions such as the internet failing in the middle of transfer or the local or remote computer shutting down, but I guess I can only find out from testing…

So without further to do, here’s my first release of zfsbackup.

Here is how I got zfs to share files on smb with no passwords (Note: I am on opensolaris snv_134)

Install the zfs cifs kernel package:

pfexec pkg install SUNWsmbs
pfexec pkg install SUNWsmbskr # this was already installed by default on my system

Reboot the system after this…

pfexec reboot

The service is disabled, let’s enable it.

$ svcs smb/server
STATE          STIME    FMRI
disabled       21:28:39 svc:/network/smb/server:default

$ svcadm enable -r smb/server # the -r recursively enables dependent services
# the following error message is fine.
svcadm: svc:/milestone/network depends on svc:/network/physical, which has multiple instances.

If you wish to join a certain workgroup (default is WORKGROUP):

smbadm join -w MyHome

Now let’s create the zfs file system:

pfexec zfs create -o casesensitivity=mixed -o nbmand=on rpool/storage

Let’s fix the permissions so that anyone and everyone could use access the file share:

pfexec chmod 777 /rpool/storage/
# must specify solaris chmod as GNU chmod does not know about solaris extended permissions

zfs set aclinherit=passthrough rpool/storage
zfs set aclmode=passthrough rpool/storage

pfexec /usr/bin/chmod -R A=everyone@:full_set:fd:allow /rpool/storage/

And let us make the share:

pfexec zfs set sharesmb=name=storage,guestok=true rpool/storage

Verify setup:

$ sharemgr show -v

Last step, map the windows guest user to a valid solaris account (if not, it’ll generate random UIDs):

pfexec idmap add winname:Guest unixuser:nobody

and the group too:

pfexec idmap add "wingroup:Domain Users" unixgroup:staff

Test the connection from another computer. Done! Thank you James, Simon, and Afshin for your help.

bashusr@solaris:~/backuppc/install$ cat

# Installation Guide for OpenSolaris version:
# SunOS 5.11 snv_134 i86pc i386 i86pc Solaris
# Written by Jonathan Chan <>
# We will be using most of the packages from CSW instead of the SunW versions.
# If you have already installed CSW or have SUNWApache, you will have to make
# adjustments to the installation. Especially if you already have CSW perl
# installed.
# NOTE: This guide/script must be run by a user with pfexec root priviledges.
# You can also run this as a bash script and hope the whole thing works :-P
set -u
set -e
#PKGADD_FLAGS="-n" # uncomment this for silent non-interactive install
#PKGGET_FLAGS="-f" # and this too.
#PKGUTIL_FLAGS="-y" # and one last one.
#### Prerequisites ####
# install pkg-get to get packages from the CSW repository
wget -O /tmp/pkg-get
pfexec pkgadd $PKGADD_FLAGS -d /tmp/pkg-get
# Install pkgutil (we don't really this util), you can use either/or pkg-get
# I'm a solaris newbie, so I'm trying both.
wget -O /tmp/pkgutil.pkg
pfexec pkgadd $PKGADD_FLAGS -d /tmp/pkgutil.pkg
# if you want to keep this path, better set it in your ~/.profile
export PATH=$PATH:/opt/csw/bin
# install packages from CSW
pfexec pkgutil $PKGUTIL_FLAGS --install gnupg lynx # dependencies for cpan
gpg --keyserver --recv-keys E12E9D2F
gpg --keyserver --recv-keys A1999E90
pfexec pkg-get $PKGGET_FLAGS -i samba_client par2cmdline apache2 apache2_manual
pfexec pkg-get $PKGGET_FLAGS -i ap2_modphp5 ap2_modperl # not used, but good modules to have for apache
# we need a compiler to compile some of the perl modules...
pfexec pkg install sunstudioexpress
# hack to make cpan work... (they hardcoded to an old version of sun studio express)
pfexec mkdir -p /opt/studio/SOS11/SUNWspro/
pfexec ln -s /opt/SunStudioExpress/bin /opt/studio/SOS11/SUNWspro/
# install perl modules from CPAN
PERL_MM_USE_DEFAULT=1 /opt/csw/bin/cpan -a
PERL_MM_USE_DEFAULT=1 pfexec /opt/csw/bin/cpan Compress::Zlib Archive::Zip File::RsyncP
#### Actual BackupPC Install ####
# Create filesystems and username
pfexec zfs create ${DIRPREFIX:1}
pfexec useradd -d $DIRPREFIX -c "BackupPC User" backuppc
# change to latest version - 3.2.0 was the latest at the time of this writing.
wget --no-check-certificate
tar zxvf BackupPC-3.2.0.tar.gz
cd BackupPC-3.2.0
# fix applications to use the correct CSW version of perl
find bin/ -type f -exec sed -i 's|/usr/bin/perl|/opt/csw/bin/perl|g' {} \;
sed -i 's|/usr/bin/perl|/opt/csw/bin/perl|g' cgi-bin/BackupPC_Admin
# fix for stop executing kill instead of running stop block.
sed -i '25s/stop/stopBackupPC/g' init.d/src/solaris-backuppc
sed -i '30s/stop/stopBackupPC/g' init.d/src/solaris-backuppc
sed -i '43s/stop/stopBackupPC/g' init.d/src/solaris-backuppc
# Correct apache configuration file since CONFDIR is chmod 750 and apache runs as nobody
sed -i "s|__CONFDIR__|${DIRPREFIX}/files/conf|g" httpd/src/BackupPC.conf
# Allow from all instead of limiting to (critical because the new versions
# of solaris use ::1 ipv6 so is not used either)
sed -i '17s/deny,allow/allow,deny/g' httpd/src/BackupPC.conf
sed -i '18s/deny/allow/g' httpd/src/BackupPC.conf
sed -i '19s/^/##/g' httpd/src/BackupPC.conf
pfexec bash -c "cat << EOF >> httpd/src/BackupPC.conf
<Directory $DIRPREFIX/files/images >
order allow,deny
allow from all
pfexec /opt/csw/bin/perl --batch \
--cgi-dir $DIRPREFIX/files/cgi-bin            \
--data-dir $DIRPREFIX/data                    \
--hostname `hostname`                         \
--html-dir $DIRPREFIX/files/images            \
--html-dir-url /BPCImages                     \
--install-dir $DIRPREFIX/files                \
--config-dir $DIRPREFIX/files/etc             \
--log-dir $DIRPREFIX/files/log
# install BackupPC Service
pfexec cp init.d/solaris-backuppc /etc/init.d/backuppc
pfexec chmod 755 /etc/init.d/backuppc
pfexec ln -s ../init.d/backuppc /etc/rc0.d/K40backuppc
pfexec ln -s ../init.d/backuppc /etc/rc2.d/S99backuppc
# Install httpd config file
pfexec mkdir $DIRPREFIX/files/conf
pfexec cp httpd/BackupPC.conf $DIRPREFIX/files/conf/BackupPC.conf
# Workaround since apache is run as nobody.
# ##WARNING## BackupPC documentation EXPLICITLY states not to do this - all
# users will be able to have access to your backup files. Please note the
# security risk of this step.
pfexec chmod a+x $DIRPREFIX/files/cgi-bin/BackupPC_Admin
pfexec bash -c 'echo "Include $DIRPREFIX/files/conf/BackupPC.conf" >> /opt/csw/apache2/etc/httpd.conf'
echo Please insert password for admin user:
pfexec /opt/csw/apache2/sbin/htpasswd -c $DIRPREFIX/files/conf/BackupPC.users admin
# start BackupPC daemon
pfexec /etc/init.d/backuppc start
# start apache
pfexec svcadm enable svc:/network/cswapache2
pfexec svcadm refresh svc:/network/cswapache2
cd ..
# delete installation files that we don't need anymore...
##rm BackupPC-3.2.0*
#firefox http://localhost/BackupPC_Admin
Also posted here: