by Bernd Schemmer, December 2008
Homepage: http://www.bnsmb.de/
Table of contents
This article describes the usage of fiocompress and dcfs to
create compressed files on UFS filesystems.
fiocompress and dcfs are new in Solaris 10 Update 6 for
SPARC.
They also exist in OpenSolaris for SPARC since Newboot for SPARC
was integrated.
Due to the fact there is only a little information about fiocompress and dcfs, this article may contain wrong or incomplete information and should be read as draft documentation by a user of the tool.
Solaris 10 Update 6 for SPARC or a current release of OpenSolaris for SPARC
Please be aware that these tools are marked with
Note that this utility is not a Committed interface. See attributes(5).
in the man page.
fiocompress(1M) man page
dcfs (7FS) man page
Porting OpenSolaris Project Indiana to SPARC - Part 2
Compression in Ramdisk - Dcfs (Moinak's blog)
dcfs is part of the Solaris package SUNWckr - Core Solaris Kernel (Root); fiocompress is part of the Solaris package SUNWcsr Core Solaris, (Root). These packages should be available on every Solaris 10 Update 6 (SPARC) or OpenSolaris installation.
The program fiocompress(1M) can be used to compress or decompress files on UFS formatted filesystems. fiocompress uses the dcfs(7FS) filesystem which uses gzip to compress or decompress the files.
Note: dcfs is a "layered filesystem" - you can not use it as primary filesystem.
The man page for fiocompress says
The primary use of fiocompress is to compress files in the boot archive.
But that's surely not the only useful usage of this tool.
The usage syntax for fiocompress is
/sbin/fiocompress -c [-m] [-b block_size] | -d {file}
The parameter -c is used to compress a file and the parameter -d is used to decompress a file (see fiocompress(1M) for a more detailed documentation for fiocompress).
Example usage:
[Fri Dec 05 22:15:07 root@sol9 /var/develop/fiocompress] # fiocompress -c SunTools.pkg SunTools.pkg.compressed [Fri Dec 05 22:15:25 root@sol9 /var/develop/fiocompress] # ls -l SunTools.pkg SunTools.pkg.compressed -rw-r--r-- 1 root root 7895552 Dec 5 22:14 SunTools.pkg -rw-r--r-- 1 root root 2030863 Dec 5 22:15 SunTools.pkg.compressed [Fri Dec 05 22:16:30 root@sol9 /var/develop/fiocompress] # file SunTools.pkg.compressed SunTools.pkg.compressed: data To decompress the file again use [Fri Dec 05 22:17:19 root@sol9 /var/develop/fiocompress] # fiocompress -d SunTools.pkg.compressed SunTools.pkg.decompressed [Fri Dec 05 22:17:26 root@sol9 /var/develop/fiocompress] # ls -l SunTools.pkg.compressed SunTools.pkg.decompressed SunTools.pkg -rw-r--r-- 1 root root 7895552 Dec 5 22:14 SunTools.pkg -rw-r--r-- 1 root root 2030863 Dec 5 22:15 SunTools.pkg.compressed -rw-r--r-- 1 root root 7895552 Dec 5 22:17 SunTools.pkg.decompressed [Fri Dec 05 22:17:44 root@sol9 /var/develop/fiocompress] # diff SunTools.pkg SunTools.pkg.decompressed
So, that's not different to the usage of gzip or any other program to compress files - so why should you use fiocompress?
The real advantage of fiocompress is hidden in the man page for fiocompress; there it says:
The following options are supported:
...
-m
Mark the compressed file for automatic decompression on read. Can be used only in conjunction with -c.
...
So, let's test this option:
[Fri Dec 05 22:21:09 root@sol9 /var/develop/fiocompress] # fiocompress -m -c SunTools.pkg SunTools.pkg.compressed_with_m [Fri Dec 05 22:21:37 root@sol9 /var/develop/fiocompress] # ls -l SunTools.pkg SunTools.pkg.compressed_with_m -rw-r--r-- 1 root root 7895552 Dec 5 22:14 SunTools.pkg -rw-r--r-- 1 root root 7895552 Dec 5 22:21 SunTools.pkg.compressed_with_m [Fri Dec 05 22:21:47 root@sol9 /var/develop/fiocompress] # du SunTools.pkg SunTools.pkg.compressed_with_m 15440 SunTools.pkg 3984 SunTools.pkg.compressed_with_m [Fri Dec 05 22:21:53 root@sol9 /var/develop/fiocompress] # file SunTools.pkg.compressed_with_m SunTools.pkg.compressed_with_m: package datastream [Fri Dec 05 22:22:00 root@sol9 /var/develop/fiocompress] # diff SunTools.pkg.compressed_with_m SunTools.pkg
Hopefully you see the difference: The dcfs filesystem adds a decompression on the fly for UFS formatted filesystems!
That's a really cool feature.
Using this feature you can compress your files and still use them without thinking about the compression also on UFS formatted filesystems.
One disadvantage of fiocompress is that it only supports read-only operations, e.g. copying a file compressed with the -m option decompresses the file, e.g.
[Fri Dec 05 22:22:09 root@sol9 /var/develop/fiocompress] # cp SunTools.pkg.compressed_with_m SunTools.pkg.compressed_with_m.copy [Fri Dec 05 22:24:25 root@sol9 /var/develop/fiocompress] # du -ks SunTools.pkg.compressed_with_m SunTools.pkg.compressed_with_m.copy 1992 SunTools.pkg.compressed_with_m 7720 SunTools.pkg.compressed_with_m.copy
From another point of view this an advantage because you do not have to think about how to decompress a file ...
And you can not edit or change a compressed file without first decompressing it. But for files that are only used in read-only mode fiocompress is really useful.
# Create the directory structure # [Fri Dec 05 22:49:32 root@sol9 /var/develop/fiocompress] # mkdir man [Fri Dec 05 22:49:35 root@sol9 /var/develop/fiocompress] # cd /usr/share/man ; find . -type d | cpio -pdum /var/develop/fiocompress/man/ 0 blocks # copy and compress the files # [Fri Dec 05 22:50:51 root@sol9 /usr/share/man] # time for i in $( find . -type f ) ; do echo fiocompress -m -c $i /var/develop/fiocompress/man/$i ; done [Fri Dec 05 22:51:34 root@sol9 /usr/share/man] # time for i in $( find . -type f ) ; do fiocompress -m -c $i /var/develop/fiocompress/man/$i ; done real 3m16.741s user 0m37.202s sys 1m39.979s # check the results # [Fri Dec 05 23:03:49 root@sol9 /usr/share/man] # du -ks /usr/share/man /var/develop/fiocompress/man 58941 /usr/share/man 28890 /var/develop/fiocompress/man # and test the new man directory tree # [Fri Dec 05 23:04:40 root@sol9 /usr/share/man] # man -M /var/develop/fiocompress/man fiocompress Reformatting page. Please Wait... done System Administration Commands fiocompress(1M) NAME fiocompress - file compression utility SYNOPSIS /sbin/fiocompress -c [-m] [-b block_size] | -d {file} DESCRIPTION The fiocompress utility is a file compression tool that works together with the dcfs(7FS) file system to perform per-file compression. You can use fiocompress to decompress a compressed file or mark a compressed file as compressed, causing automatic decompression on read. The primary use of fiocompress is to compress files in the boot archive. Note that this utility is not a Committed interface. See attributes(5). OPTIONS The following options are supported: -b block_size Specify a block size for compression. The default block size is 8192.i -c Compress the specified file. -d Decompress the specified file. -m Mark the compressed file for automatic decompression on read. Can be used only in conjunction with -c. EXIT STATUS 0
# Compress the ISO image # [Sat Dec 06 09:44:58 root@sol9 /] # time fiocompress -m -c /export/iso/Solaris10/sol-10-u5-doc-ga-dvd.iso /var/daten/sol-10-u5-doc-ga-dvd.iso.compressed real 1m23.849s user 1m15.220s sys 0m4.894s # check the result # [Sat Dec 06 09:47:05 root@sol9 /] # ls -l [Sat Dec 06 09:55:16 root@sol9 /] # ls -l /export/iso/Solaris10/sol-10-u5-doc-ga-dvd.iso /var/daten/sol-10-u5-doc-ga-dvd.iso.compressed -rw-rw-r-- 1 root root 426311680 Apr 5 2008 /export/iso/Solaris10/sol-10-u5-doc-ga-dvd.iso -rw-r--r-- 1 root root 426311680 Dec 6 09:47 /var/daten/sol-10-u5-doc-ga-dvd.iso.compressed [Sat Dec 06 09:55:23 root@sol9 /] # du -ks /export/iso/Solaris10/sol-10-u5-doc-ga-dvd.iso /var/daten/sol-10-u5-doc-ga-dvd.iso.compressed 416536 /export/iso/Solaris10/sol-10-u5-doc-ga-dvd.iso 305624 /var/daten/sol-10-u5-doc-ga-dvd.iso.compressed # Use the compressed ISO image # [Sat Dec 06 09:55:28 root@sol9 /] # lofiadm -a /var/daten/sol-10-u5-doc-ga-dvd.iso.compressed /dev/lofi/24 [Sat Dec 06 09:56:01 root@sol9 /] # mkdir /tmp/sol10u5 [Sat Dec 06 09:56:07 root@sol9 /] # mount -o ro -F hsfs /dev/lofi/24 /tmp/sol10u5 [Sat Dec 06 09:56:24 root@sol9 /] # ls /tmp/sol10u5 Copyright css graphics index.html pdf
This is a bug in the current version of fiocompress; see the blog entry Porting OpenSolaris Project Indiana to SPARC - Part 2 (Update 17.04.2012: The blog entry is not there anymore ...) for a workaround for this bug.
The bugid for this bug is 6724205.
There seems to be a size restriction for fiocompress or dcfs, example:
[Sat Dec 06 10:01:51 root@sol9 /] # ls -l /export/iso/Solaris10/sol-10-u6-ga1-sparc-dvd.iso -rw-r--r-- 1 root root 2661416960 Oct 28 20:12 /export/iso/Solaris10/sol-10-u6-ga1-sparc-dvd.iso [Sat Dec 06 10:01:52 root@sol9 /] # du -h /export/iso/Solaris10/sol-10-u6-ga1-sparc-dvd.iso 2.5G /export/iso/Solaris10/sol-10-u6-ga1-sparc-dvd.iso [Sat Dec 06 10:02:23 root@sol9 /] # fiocompress -m -c /export/iso/Solaris10/sol-10-u6-ga1-sparc-dvd.iso /var/daten/sol-10-u6-ga1-sparc-dvd.iso fiocompress: cannot open /export/iso/Solaris10/sol-10-u6-ga1-sparc-dvd.iso - Value too large for defined data type