Software Distribution
Using the
ESP Package Manager

2 March 2001

Michael Sweet, Easy Software Products

ESP-004-20010302

Contents

Introduction

Software distribution under UNIX/Linux can be a challenge, especially if you ship software for more than one operating system. Every operating system provides its own software packaging tools and each has unique requirements or implications for the software development environment.

The ESP Package Manager ("EPM") is one solution to this problem. Besides its own "portable" distribution format, EPM also supports the generation of several vendor-specific formats. This allows you to build software distribution files for almost any UNIX/Linux operating system from the same sources.

History and Evolution

When Easy Software Products was founded in 1993, we originally shipped software only for the SGI IRIX operating system. In 1997 we added support for Solaris followed by HP-UX support in 1998, each time building a new set of packaging files for each operating system and supported processor. While this worked, it also meant that we had to keep all of the packaging files synchronized manually. Needless to say, this process was far from perfect and we had more than one distribution that was not identical on all operating systems.

As we began developing the Common UNIX Printing System ("CUPS") in 1998, our goal was to add support for two additional operating systems: Linux and Compaq Tru64 UNIX. Clearly we had to change how we did software distributions.

The first version of EPM was released in 1999 and supported so-called "portable" software distributions that were not tied to any particular operating system or packaging software. Due to popular demand, we added support for specific packaging formats in the second major release of EPM, allowing the generation of portable or "native" distributions from one program.

Existing Software Packaging Systems

As we looked for a solution to our problem, we naturally investigated the existing open-source packaging systems. Under Linux, we looked at the RedHat Package Manager ("RPM") and Debian packaging software ("dpkg" and "dselect"). For the commercial UNIX's we looked at the vendor-supplied packaging systems. The following table shows the results of our investigation:

Format Operating Systems1 Binaries? Cross- Platform? Patches? Upgrades? Conflicts? Requires? Replaces? Config Files? Map Files?
Debian Corel Linux
Debian GNU/Linux
Yes Yes2 No Yes Yes Yes No Yes No
depot HP-UX Yes No Yes Yes Yes Yes No Yes Yes
inst IRIX Yes No Yes Yes Yes Yes Yes Yes Yes
pkg Solaris Yes No Yes No Yes Yes No No Yes
RPM Mandrake
RedHat
SuSE
TurboLinux
Yes Yes2 No Yes Yes Yes No Yes No

  1. Standard packaging system for named operating systems.
  2. These packaging systems are cross-platform but require the package management utilities to be installed on the platform before installing the package.

As you can see, none of the formats supported every feature we were looking for. One common fault of all these formats is that they do not support a common software specification file format. That is, making a Debian software distribution requires significantly different support files than required for a Solaris pkg distribution. This makes it extremely difficult to manage distributions for multiple operating systems.

All of the package formats support binary distributions. The RPM and Debian formats also support source distributions that specifically allow for recompilation and installation. Only the commercial UNIX formats support patch distributions - you have to completely upgrade a software package with RPM and Debian. All but the Solaris pkg format allow you to upgrade a package without removing the old version first.

When building the software packages, RPM and Debian force you to create the actual directories, copy the files to those directories, and set the ownerships and permissions. You essentially are creating a directory for your software that can be archived in the corresponding package format. To ensure that all file permissions and ownerships are correct, you must build the distribution as the root user, introducing potential security risks and violating many corporate security policies. It can also make building distributions difficult when dynamic data such as changing data files or databases is involved.

The commercial UNIX formats use software list files that map source files to the correct directories and permissions. This allows for easier delivery of dynamic data, configuration management of what each distribution actually contains, and eliminates security issues with special permissions and building distributions as the root user. Using the proprietary format also has the added benefit of allowing for software patches and using the familiar software installation tools for that operating system. The primary disadvantage is that the same distributions and packaging software cannot be used on other operating systems.

Design Goals of EPM

Our goals for the first release of EPM were fairly simple:

  1. Build binary software distributions using a common software specification format. The same distribution files must work for all operating systems.
  2. Optionally strip executable files before packaging.
  3. Map source files and directories to the installation locations and permissions.
  4. Support configuration files and boot-time initialization scripts.
  5. Automatically backup existing files when a product replaces an existing file.
  6. Provide installation and removal scripts that will work on all systems without additional software.
  7. Support automatic upgrades to newer versions of a product using full or (smaller) patch distributions.
  8. Support software package conflicts, requirements, and replacements.
  9. Support network installations where /usr is shared amongst client machines.
  10. Make it all open-source under the GNU GPL.

For EPM 2.0 we added the following goals:

  1. Support portable and vendor package formats from the same specification files. Make the interface easily extended to support new formats as they are implemented.
  2. Provide a simple GUI installer that can be included with portable distributions, like the popular InstallShield software in the Microsoft Windows world.
  3. Support variable substitution in filenames to allow for easier relocation and/or naming of files in the distribution.
  4. Support wildcards in source filenames.

As you can see, the focus has been on making the binary software distribution process as easy as possible. Supporting source code distributions was not a goal since most RPM and Debian source distributions are little more than wrapping around a compressed tar file containing the source files and a configure script.

EPM List Files

EPM reads one or more software "list" files that describe a single software package. Each list file contains one or more lines of ASCII text containing product or file information. Comments start with the # character.

Product Information

The following is an excerpt from the EPM software list file (epm.list) provided with the software:

As you can see, product information lines (as well as other EPM directives) start with the percent (%) character. The attribute name is followed by the value separated by whitespace.

Variables

EPM imports the current environment variables for use in your list file. You can also define new variable in the list file or on the command-line when running EPM.

Variables are defined by starting the line with the dollar sign ($) followed by the name and value:

Variable substitution is performed when the variable is defined, so be careful with the ordering of your variable definitions.

Also, any variables you specify in your list file will be overridden by variables defined on the command-line or in your environment, just like with make. This can be a useful feature or a curse, depending on your choice of variable names.

As you can see, variables are referenced using the dollar sign ($). As with most shells, variable names can be surrounded by curly braces (${variable}) to explicitly delimit the name.

If you need to insert a $ in a filename or a script, use $$:

Files

Following the product information is the list of files to be included in the distribution:

The %system directive is one of two conditional statements that can be included in software list files. In this case we are installing the man pages in one of two directories depending on the operating system.

Plain files are indicated by the f at the beginning of each line. Directories, configuration files, initialization scripts, and symbolic links are specified using the letters d, c, i, and l, respectively. Files that should be included in a patch upgrade use uppercase letters instead of lowercase.

The remaining data fields on the line are the file permission bits, the owner, the group, the destination filename, and the source filename. The source and destination filenames can contain references to environment variables and variables passed on the EPM command-line. For example, the man page files can reference the mandir variable to relocate the installed man pages instead of using the %system conditional directive:

As you can see, variables are referenced using the dollar sign ($). As with most shells, variable names can be surrounded by curly braces (${variable}) to explicitly delimit the name.

The source field can also use shell wildcards to include multiple files on a single line:

Directories

Directories use a source file of -:

Configuration Files

Configuration files use the same format as regular files; if the configuration file exists when the software is installed, then the new configuration file is stored as filename.N so it can be merged with the old configuration file manually. This is one of the configuration files supplied with the CUPS distribution:

Intialization Scripts

Initialization scripts are scripts or programs that are run at boot time, usually to start daemon programs. EPM automatically determines the location of initialization files at install time, so it can handle the different locations used by various Linux distributions. This is the initialization script used by CUPS:

This specifies that the root name of the initialization script is cups, and the file cups.sh should be used as the source for the script. EPM creates links for startup and shutdown at run levels 0, 2, 3, and 5 as appropriate.

Symbolic Links

Symbolic links use the source field as the link value. That is, if you want to provide a symbolic link from gzip to gunzip in /usr/bin you'd use:

Conditional Directives

As stated before, the %system directive can be used to conditionally include specific files like the man pages in the EPM distribution.

The %system directive can match or not match specific operating system names or versions. The operating system name is the name reported by uname in lowercase, while the operating system version is the major and minor version number reported by uname -r:

%system irix
Only include the following files when building a distribution for the IRIX operating system.
%system linux-2.0
Only include the following files when building a distribution for Linux 2.0.x.
%system !irix !linux-2.0
Only include the following files when building a distribution for operating systems other than IRIX and Linux 2.0.x.

The special name all is used to match all operating systems:

For format-specific files, the %format directive can be used:

%format rpm
Only include the following files when building an RPM distribution.
%format !rpm
Only include the following files when not building an RPM distribution.x.
%format all
Include the following files for all types of distributions.

Conflicts, Replaces, and Requires

Software conflicts and requirements are specified using the %incompat and %requires directives. If your software replaces another package, you can specify that using the %replaces directive (%replaces is silently mapped to %conflicts when the distribution format does not support package replacement.)

Dependencies are specified using the package name:

or the filename:

Package dependencies are currently enforced only for the same package format, so a portable distribution that requires package "foobar" will only look for an installed "foobar" package in portable format.

Filename dependencies are only supported by the Debian, portable, and RPM distribution formats.

Building Distributions with EPM

To build a distribution called "epm" from this list file, simply run:

EPM will automatically load the file epm.list for the list file; to specify the file directly, use:

The first argument is the name of the package, while the second is the list filename. The result will be a gzip'd tar file containing the distribution files and installation script needed to install the software.

Variables are specified using using the equal sign (=) on the command-line or in the list file ($name=value); there is no limit on the number of variables you can specify:

EPM can also produce vendor-specific distributions using the -f option:

The format option can be one of:

Everything in the software list file stays the same - you just use the -f option to select the format. For example, to build an RPM distribution of EPM, type:

The result will be an RPM distribution file instead of the portable distribution file.

Further Reading

EPM supports several command-line options and many directives. For more information see the EPM man page on-line with:

You can also view the man page and other examples on-line at:

Summary

EPM provides a common interface to build binary software distributions in a variety of formats. Since distributions are built using the same software list file for all formats, packages built using EPM likely will be more consistent across platforms and will require less work to maintain. Since EPM is designed to support multiple distribution formats, it can be easily extended to support new formats, further reducing the amount of work needed to support multiple platforms.

Since EPM-generated portable and RPM distributions automatically detect where initialization scripts should be installed, EPM is especially useful for producing Linux software packages that need to run under multiple Linux distributions.

EPM is provided under the GNU General Public License. The EPM home page is at:

Easy Software Products also provides commercial support for EPM. Please consult the EPM home page for more information.