11 December 2000
Michael Sweet, Easy Software Products
ESP-004-20001211
Software distribution under UNIX/Linux can be a challenge, especially if you ship software for more than one operating system. Every operating system provides its own software packaging tools and each has unique requirements or implications for the software development environment.
The ESP Package Manager ("EPM") is one solution to this problem. Besides its own "portable" distribution format, EPM also supports the generation of several vendor-specific formats. This allows you to build software distribution files for almost any UNIX/Linux operating system from the same sources.
When Easy Software Products was founded in 1993, we originally shipped software only for the SGI IRIX operating system. In 1997 we added support for Solaris followed by HP-UX support in 1998, each time building a new set of packaging files for each operating system and supported processor. While this worked, it also meant that we had to keep all of the packaging files synchronized manually. Needless to say, this process was far from perfect and we had more than one distribution that was not identical on all operating systems.
As we began developing the Common UNIX Printing System ("CUPS") in 1998, our goal was to add support for two additional operating systems: Linux and Compaq Tru64 UNIX. Clearly we had to change how we did software distributions.
The first version of EPM was released in 1999 and supported so-called "portable" software distributions that were not tied to any particular operating system or packaging software. Due to popular demand, we added support for specific packaging formats in the second major release of EPM, allowing the generation of portable or "native" distributions from one program.
As we looked for a solution to our problem, we naturally investigated the existing open-source packaging systems. Under Linux, we looked at the RedHat Package Manager ("RPM") and Debian packaging software ("dpkg" and "dselect"). For the commercial UNIX's we looked at the vendor-supplied packaging systems. The following table shows the results of our investigation:
Format | Operating Systems1 | Binaries? | Cross- Platform? | Patches? | Upgrades? | Conflicts? | Requires? | Replaces? | Config Files? | Map Files? |
---|---|---|---|---|---|---|---|---|---|---|
Debian | Corel Linux Debian GNU/Linux |
Yes | Yes2 | No | Yes | Yes | Yes | No | Yes | No |
depot | HP-UX | Yes | No | Yes | Yes | Yes | Yes | No | Yes | Yes |
inst | IRIX | Yes | No | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
pkg | Solaris | Yes | No | Yes | No | Yes | Yes | No | No | Yes |
RPM | Mandrake RedHat SuSE TurboLinux |
Yes | Yes2 | No | Yes | Yes | Yes | No | Yes | No |
As you can see, none of the formats supported every feature we were looking for. One common fault of all these formats is that they do not support a common software specification file format. That is, making a Debian software distribution requires significantly different support files than required for a Solaris pkg distribution. This makes it extremely difficult to manage distributions for multiple operating systems.
All of the package formats support binary distributions. The RPM and Debian formats also support source distributions that specifically allow for recompilation and installation. Only the commercial UNIX formats support patch distributions - you have to completely upgrade a software package with RPM and Debian. All but the Solaris pkg format allow you to upgrade a package without removing the old version first.
When building the software packages, RPM and Debian force you to create the actual directories, copy the files to those directories, and set the ownerships and permissions. You essentially are creating a directory for your software that can be archived in the corresponding package format. To ensure that all file permissions and ownerships are correct, you must build the distribution as the root user, introducing potential security risks and violating many corporate security policies. It can also make building distributions difficult when dynamic data such as changing data files or databases is involved.
The commercial UNIX formats use software list files that map source files to the correct directories and permissions. This allows for easier delivery of dynamic data, configuration management of what each distribution actually contains, and eliminates security issues with special permissions and building distributions as the root user. Using the proprietary format also has the added benefit of allowing for software patches and using the familiar software installation tools for that operating system. The primary disadvantage is that the same distributions and packaging software cannot be used on other operating systems.
Our goals for the first release of EPM were fairly simple:
/usr
is shared amongst client machines.
For EPM 2.0 we added the following goals:
As you can see, the focus has been on making the binary software distribution process as easy as possible. Supporting source code distributions was not a goal since most RPM and Debian source distributions are little more than wrapping around a compressed tar file containing the source files and a configure script.
EPM reads one or more software "list" files that describe a
single software package. Each list file contains one or more
lines of ASCII text containing product or file information.
Comments start with the #
character.
The following is an excerpt from the EPM software list file
(epm.list
) provided with the software:
# Product information %product ESP Package Manager %copyright 1999-2000 by Easy Software Products, All Rights Reserved. %vendor Easy Software Products %license COPYING %readme README %description Universal software packaging tool for UNIX. %version 2.2
As you can see, product information lines (as well as other
EPM directives) start with the percent (%
)
character. The attribute name is followed by the value separated
by whitespace.
Following the product information is the list of files to be included in the distribution:
# Executables %system all f 0555 root sys /usr/bin/epm epm f 0555 root sys /usr/lib/epm/setup setup f 0444 root sys /usr/share/doc/epm/README README f 0444 root sys /usr/share/doc/epm/COPYING COPYING f 0444 root sys /usr/share/doc/epm *.html # Man pages %system irix f 0444 root sys /usr/share/catman/u_man/cat1/epm.1 epm.1 %system !irix f 0444 root sys /usr/man/man1/epm.1 epm.man
The %system
directive is one of two conditional
statements that can be included in software list files. In this
case we are installing the man pages in one of two directories
depending on the operating system.
Plain files are indicated by the f
at the
beginning of each line. Directories, configuration files,
initialization scripts, and symbolic links are specified using
the letters d
, c
, i
, and
l
, respectively. Files that should be included in a
patch upgrade use uppercase letters instead of lowercase.
The remaining data fields on the line are the file permission
bits, the owner, the group, the destination filename, and the
source filename. The source and destination filenames can
contain references to environment variables and variables passed
on the EPM command-line. For example, the man page files can
reference the mandir
variable to relocate the
installed man pages instead of using the %system
conditional directive:
# Man pages f 0444 root sys $mandir/cat1/epm.1 epm.1 f 0444 root sys $mandir/man1/epm.1 epm.man
As you can see, variables are referenced using the dollar
sign ($
). As with most shells, variable names can
be surrounded by curly braces (${variable}
) to
explicitly delimit the name.
The source field can also use shell wildcards to include multiple files on a single line:
f 0444 root sys /usr/share/doc/epm *.html
Directories use a source file of -
:
d 0755 root sys /foo/bar/dir -
Configuration files use the same format as regular files; if
the configuration file exists when the software is installed,
then the new configuration file is stored as
filename.N
so it can be merged with the old
configuration file manually. This is one of the configuration
files supplied with the CUPS distribution:
c 0644 root sys /etc/cups/cupsd.conf conf/cupsd.conf
Initialization scripts are scripts or programs that are run at boot time, usually to start daemon programs. EPM automatically determines the location of initialization files at install time, so it can handle the different locations used by various Linux distributions. This is the initialization script used by CUPS:
i 0555 root sys cups cups.sh
This specifies that the root name of the initialization
script is cups
, and the file cups.sh
should be used as the source for the script. EPM creates links
for startup and shutdown at run levels 0, 2, 3, and 5 as
appropriate.
Symbolic links use the source field as the link value. That
is, if you want to provide a symbolic link from
gzip
to gunzip
in
/usr/bin
you'd use:
l 0555 root sys /usr/bin/gunzip gzip
As stated before, the %system
directive can be used
to conditionally include specific files like the man pages in the EPM
distribution.
The %system
directive can match or not match
specific operating system names or versions. The operating
system name is the name reported by uname
in
lowercase, while the operating system version is the major and
minor version number reported by uname -r
:
%system irix
%system linux-2.0
%system !irix !linux-2.0
The special name all
is used to match all operating systems:
%system all
For format-specific files, the %format
directive can be
used:
%format rpm
%format !rpm
%format all
Software conflicts and requirements are specified using the
%incompat
and %requires
directives. If
your software replaces another package, you can specify that
using the %replaces
directive (%replaces
is silently mapped to %conflicts
when the distribution
format does not support package replacement.)
Dependencies are specified using the package name:
%requires foobar %incompat foobar %replaces foobar
or the filename:
%requires /usr/lib/libfoobar.so %incompat /usr/lib/libfoobar.so.1.2
Package dependencies are currently enforced only for the same package format, so a portable distribution that requires package "foobar" will only look for an installed "foobar" package in portable format.
Filename dependencies are only supported by the Debian, portable, and RPM distribution formats.
To build a distribution called "epm" from this list file, simply run:
epm epm
EPM will automatically load the file epm.list
for
the list file; to specify the file directly, use:
epm epm epm.list
The first argument is the name of the package, while the second is the list filename. The result will be a gzip'd tar file containing the distribution files and installation script needed to install the software.
Variables are specified using using the equal sign (=
);
there is no limit on the number of variables you can specify:
epm mandir=/foo/bar/man epm epm.list
EPM can also produce vendor-specific distributions using the
-f
option:
epm -f format ...
The format option can be one of:
depot
or swinstall
- HP-UX software
distribution.
dpkg
- Debian software distribution.
inst
or tardist
- IRIX software
distribution.
native
- "Native" software distribution (RPM,
INST, DEPOT, PKG, etc.) for the platform.
pkg
- Solaris software distribution.
portable
- Portable software distribution (default).
rpm
- RedHat software distribution.
Everything in the software list file stays the same - you just use the
-f
option to select the format. For example, to build an RPM
distribution of EPM, type:
epm -f rpm epm
The result will be an RPM distribution file instead of the portable distribution file.
EPM supports several command-line options and many directives. For more information see the EPM man page on-line with:
man epm
You can also view the man page and other examples on-line at:
http://www.easysw.com/epm/documentation.html
EPM provides a common interface to build binary software distributions in a variety of formats. Since distributions are built using the same software list file for all formats, packages built using EPM likely will be more consistent across platforms and will require less work to maintain. Since EPM is designed to support multiple distribution formats, it can be easily extended to support new formats, further reducing the amount of work needed to support multiple platforms.
Since EPM-generated portable and RPM distributions automatically detect where initialization scripts should be installed, EPM is especially useful for producing Linux software packages that need to run under multiple Linux distributions.
EPM is provided under the GNU General Public License. The EPM home page is at:
http://www.easysw.com/epm
Easy Software Products also provides commercial support for EPM. Please consult the EPM home page for more information.