Generalization for “Speed-up system by SquashFs”


Please note that this blog has been moved.

Now it has its own domain: mynixworld.info🙂

If you want to read the latest version of this article (recommended) please click here and I open the page for you.

In the previous post I wrote about how to speed-up your (Gentoo) system by using the SquashFS filesystem.

I had described shortly an experiment of compressing the /usr folder into a SquashFS image (which will become smaller and will boost your system) and using that image instead of the /usr original folder.

The steps I had used for /usr example can be generalized into a more abstract version which in turn can be used for almost any /folder you want to squash.

Basically the concept is the following:

  • create a “repository” where you will save all your SquashFS images (ex: /squashed/{usr,sbin,opt} etc)
  • optional : create manually the SquashFS images for all your folders you want to squash (ex: /usr , /sbin, /opt, etc)
  • we will have a generic runscript /etc/init.d/squashd that will help us to mount/unmount the SquashFS images at boot/shutdown time
  • for every /<folder> we want to squash and mount/unmount its image at boot/shutdown time:
    • we’ll have a dedicated symlink in /etc/init.d/ that will point to the /etc/init.d/squashd generic runscript;
    • for those dedicated symlinks we must to have a corresponding /etc/conf.d/squash-<folder> configuration files which will help us to specify how the generic squashd script should work for our particular <folder>.
    • we’ll have to create a corresponding service with the help of rc-update

So, what are the steps?

Step 1: create a /squashed folder in your root.

Step 2: for every <folder> you want to squash create a corresponding subdirectory in /squashed folder

ex: /squashed/{usr,sbin,opt} etc

Step 3A: create the generic /etc/init.d/squashd runscript as following:

#!/sbin/runscript
# Copyright 1999-2006 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $
#
# /etc/init.d/squashd allows efficient compression of Gentoo /$IMGNAME
#
# It requires support for the loop device and squashfs enabled in the kernel,
# module autoloading is also *highly* recommended.
# sys-fs/squashfs and sys-fs/aufs are necessary for read-write support.
#
# Author: Mathias Laurin <mathias_laurin@users.sourceforge.net>
# 2006-11-28, v.0.1.5(4)
# 2009-02-24, v.0.1.6(1) Weedy <weedy2887@gmail.com>
# 2009-03-20, v.0.1.7(1) j0inty <j0inty@stollfuss.net>
# 2009-07-10, v.0.1.8(1) j0inty
# 2009-09-01, v.0.1.9(1) nall <soir@fuzzysock.net>
# 2012-02-25. v.0.1.10(2) Eugen <eugenmihailescux@gmail.com>

#source /etc/make.globals
source /etc/make.conf

SQFS_CUR="$SQFS_DIRNAME/$IMGNAME.sqfs" # the squashed filesystem's image
SQFS_NEW="$SQFS_DIRNAME/$IMGNAME-current.sqfs" # the current using version
SQFS_OLD="$SQFS_DIRNAME/$IMGNAME-old.sqfs" # the old version of image file

MNTDIR_RW="" # where to mount the rw portion of squashed filesystem
DEF_RW="/dev/shm/.$IMGNAME-rw" # if MNTDIR_RW not defined then use default one

SQFS_OPTS="-all-root -no-duplicates" # options used for mounting SquashFS

depend() {
	need localmount modules $DEPENDS
	before $BEFORE
}

check_support() {
	if ! [ -w /dev/loop0 ] ; then
		eerror "ERROR: loopback support is not available."
		return 1
	fi
	if ! [[ $(grep -s $'\taufs$' /proc/filesystems) ]] ; then
		eerror "ERROR: aufs filesystem support is not available."
		return 1
	fi
	if ! [[ $(grep -s $'\tsquashfs$' /proc/filesystems) ]] ; then
		eerror "ERROR: squashfs filesystem support is not available."
		return 1
	fi
	return 0
}

makeImage() {
	mksquashfs $MNTDIR_RO $SQFS_NEW $SQFS_OPTS -b 65536 # 2>/dev/null
	retval=$?
	ln -sf $SQFS_NEW $SQFS_CUR
 	eend $retval
}

start() {
	check_support || return 1
	if [ -f "$SQFS_CUR" ]; then
		ebegin "SQFS-$IMGNAME: Mounting $SQFS_CUR read-only squashfs image as $MNTDIR_RO"
		mount -t squashfs -o loop,ro $SQFS_CUR $MNTDIR_RO
		retval=$?
		[ $retval -ne 0 ] && return $retval
	else
		if [ ! -d $MNTDIR_RO ]; then
			ebegin "SQFS-$IMGNAME: $SQFS_CUR looks empty or corrupted, using the original"
		        return 2
		fi
	fi

	ebegin "Mounting $MNTDIR_RW read-write with aufs as $MNTDIR_RO"
	if [ ! $MNTDIR_RW ] ; then
		einfo "  will be mounted in tmpfs (RAM)"
		MNTDIR_RW="${DEF_RW}"
	fi
	[ -d $MNTDIR_RW ] || mkdir -p $MNTDIR_RW
	mount -t aufs -o udba=reval,br=$MNTDIR_RW:$MNTDIR_RO aufs $MNTDIR_RO
	eend $?
}

stop() {
	ebegin "SQFS-$IMGNAME: Stopping and unmounting"
	[ ! $MNTDIR_RW ] && MNTDIR_RW="${DEF_RW}"
	rw_size=$(du -s --exclude={.w*$EXCLUDE} $MNTDIR_RW | cut -f 1)
	if [ $rw_size -gt 4 ]; then
		einfo "  Changes detected ($rw_size bytes), updating image."
		mv -f $SQFS_NEW $SQFS_OLD
		makeImage
		rm -f $SQFS_OLD
	else
		einfo "  No changes detected, skipping update."
		eend 0
	fi

	einfo "  Unmounting the $MNTDIR_RO aufs/squash image"
	umount -l -t aufs  $MNTDIR_RO
	umount -l -t squashfs $MNTDIR_RO
	rm -rf $MNTDIR_RW
	eend 0
}

Step 3B: for every <folder> you want to squash create a symlink from /etc/init.d/squashd to /etc/init.d/squash-<folder>

ex: ln -s /etc/init.d/squashd /etc/init.d/squash-<folder>

Step 4: for every <folder> you want to squash create a /etc/conf.d/squash-<folder> configuration file having the following content:

# /etc/conf.d/squash-<folder>

# SQFS_DIRNAME points to the directory that will contain the sqfs
# images, recommended value is /squashed/<folder>
SQFS_DIRNAME="/squashed/<folder>"

# Leave MNTDIR_RW empty for use with tmpfs, a ram-based filesystem,
# This is recommended unless you are short of RAM
MNTDIR_RW=""

# MNTDIR_RO points to the directory that will contain the mounted
# version of sqfs image, recommended valus is /<folder>
MNTDIR_RO="/<folder>"

# IMGNAME is the name used by the sqfs image filesystem
# Recommended value is the name of the MNTDIR_RO, ex: <folder>
IMGNAME="<folder>"

# Exclude the following pattern from MNTDIR_RW when searching
# for changes that have to be written to the sqfs image.
# If EXCLUDE is not empty then first character of it should be
# comma.
# Ex: EXCLUDE=",*/rc/init.d/stopping" will exclude the
# */rc/init.d/stopping folder/file when trying to determine
# whether or not the read-write content of /dev/shm/.<folder>-rw
# has been changed.
EXCLUDE=""

# Write space-separated the list of the daemons that
# this one depends on and should be started before this one
DEPENDS=""

Note: in the above configuration file replace the appearance of <folder> with the full path of your particular folder.

Step 5: for all SquashFS images of <folders> you want to be automatically mounted/unmounted at boot/shutdown time create a service with the default/boot runlevel as following:

rc-update add squash-<folder> <runlevel>

For exemple, I had created squash-usr service with the boot runlevel, squash-opt respectively squash-sbin with the default runlevel.

Note: when starting the services they will check if the SquashFS image of <folder> is already created and if not then it will take care by itself on that, so it will be not necessary to do it manually by yourself (unless you want that for particular reasons).

Warning: before doing anything backup your data ;o)

About Eugen Mihailescu

Always looking to learn more about *nix world, about the fundamental concepts of arithmetic, algebra and geometry. I am also passionate about programming, database and systems administration.
This entry was posted in linux, shell and tagged , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s