penv - the persistent environment utility

$Ringlet: c/misc/penv/doc/en_US.ISO8859-1/articles/penv/penv.sgml,v 1.4 2002/03/22 07:50:36 roam Exp $

This article describes the goals, usage and development history of penv, a utility to manage persistent per-directory environment settings.


Table of Contents
1. What is penv?
2. penv development history
3. Building and installing penv
4. Using penv
5. Compatibility and interoperability
6. A brief look at the penv source tree

1. What is penv?

penv is a utility which helps manage persistent per-directory environment settings. In other words, it saves you the trouble of setting or specifying the same environment variables over and over again when executing programs in a specific directory. The main reason for the initial development of penv was its use with the FreeBSD Ports Collection , but it may be used for other day-to-day tasks, too.

For its operation, penv needs to run an ``environment processor'' - a program that sets up the environment according to the contents of files in a given directory. By default, penv uses for its environment processor the envdir command from Dr. Dan J. Bernstein's daemontools package, available at http://cr.yp.to/daemontools.html.


2. penv development history

For the penv development history, please see the ChangeLog in HTML or text format.


3. Building and installing penv

The two main ways to install penv are building it from source and from a FreeBSD port.


3.2. Building and installing penv from source

To install penv from its source distribution, follow this procedure:

  1. Acquire the source distribution

    The penv source distribution may be obtained from its home page, http://devel.ringlet.net/sysutils/penv/. The current version of penv is 1.2p1; a source tarball may be obtained directly from http://devel.ringlet.net/sysutils/penv/penv-1.2p1.tar.gz.

    Another way to acquire the penv source distribution would be to access Ringlet's CVS repository and check out the penv module. However, for the present, the Ringlet CVS repository is not available online for anonymous access.

  2. Unpack the penv source

    If you have fetched the penv tar/gzip source archive, you will need to unpack it using the command:

        % tar -zxf /path/to/penv-1.2p1.tar.gz
    
  3. Configure the penv build settings

    This and the following steps should be performed from within the penv source directory. If you have obtained the penv sources via CVS, this directory would be named simply penv/. If you have obtained a tar/gzip source archive, the directory will be named penv-1.2p1/.

    Edit the appropriate Makefile for your system. If you are on a GNU/Linux system, or you are using GNU make on some other system, you need to edit the GNUmakefile. If you are using a BSD version of make or the pmake portable version, you need to edit the Makefile.

    There are two build-time configuration options that you may enable by uncommenting the lines that add them to the CFLAGS_COMPAT make variable:

    • HAVE_STRLCPY

      Use this if your C library (libc) provides a strlcpy(3) function. Most BSD-derived operating systems do.

    • HAVE_FGETLN

      Use this if your C library (libc) provides a fgetln(3) function. Most BSD-derived operating systems do.

  4. Build penv

    Once the configuration step is complete, it is time for the actual penv build. All you need to do is issue the following command:

        % make
    
  5. Install penv

    We are almost there! :) penv has been successfully built, now all you need is to install it. Just like with many other programs, this is done with the following command:

        # make install
    

    Note: You may need to obtain root privileges to install penv in a location outside your home directory or the system /tmp directory.

And that is all there is to it! :)


4. Using penv

There are three sides to using penv - configuration files, defining environment settings for a particular directory and invoking programs with those settings. For a description of the penv command-line options and the commands that may be specified with the -c option, please see the penv manual page. You may also want to look at some examples of using penv.

penv stores the environment settings for each directory into files located in the corresponding ``environment directory''. Each environment variable is stored in its own file, according to the directory layout format used by the envdir program. By default, the environment directories reside under a common ``base directory'' - /var/db/penv/ by default. The name of the environment directory is formed by taking the last components of the name of the current working directory and appending those to the base directory name. By default, the last two components are used. Thus, if the current working directory is /usr/ports/databases/mysql323-server/, then the directory where penv will store its settings will be /var/db/penv/databases/mysql323-server/. Both the base directory name and the number of path components used to construct the environment directory name may be overridden in the penv.conf file - see the global configuration section.


4.1. Configuration - the penv.conf and ~/.penvrc files.

For its operation, penv needs some information that may be defined on the command line, in environment variables or in configuration files. There are two types of files: a global configuration file named penv.conf, residing in /usr/local/etc/ by default, and a per-user one, named .penvrc, in each user's home directory. The order in which configuration options are looked for is command line, environment settings, per-user configuration file, global configuration file. A sample configuration file is included with the penv distribution.

The syntax of the penv configuration files is simple - each line is either a blank line, a comment starting with the # character, or a variable definition. A variable definition line is of the form varname=value and the variables that may be defined are:

  • basedir (string) - the base for forming the environment directory name. This variable may be overridden by the -D command-line option or the PENV_BASEDIR environment variable. The default value is /var/db/penv/.

  • depth (integer) - the number of components to be taken from the name of the current working directory to obtain the environment directory name. This variable may be overridden by the -d command-line option or the PENV_DEPTH environment variable. The default value is 2.

  • envdir_p (string) - the name (full path) of the environment processor to use. This option may be overridden by the PENV_ENVDIR_P environment variable. The default value is /usr/local/bin/envdir.


4.2. Defining environment settings

So, you want to define some environment variables that will be used every time some command is run in a certain directory. Nothing easier than that! :)

  1. Change to the directory in question

    While using penv, keep in mind that it only affects the environment settings of your current working directory. Thus, both while defining settings and running commands, you need to ``be'' in the directory for which you want those settings to pertain.

  2. Create the penv configuration directory

    penv keeps every environment variable you define in a separate file in a configuration directory. Thus, for each directory that you want to define settings for, you need to create a configuration directory. This is done by the command:

        % penv -c mkdir
    

    Note: You may use the -D basedir or -d depth command-line options to specify the location and name of the penv configuration directory; see the penv manual page for more information. In general, though, the above command will be just enough.

  3. Define the environment variables

    With penv, you may specify two kinds of environment changes: defining a variable with a certain value and undefining a variable (removing it from the environment).

    To define a variable and give it a certain value, use:

        % penv -S varname=value
    

    To specify that a variable should be removed from the environment, use:

        % penv -S varname
    

    This may raise a question: how is a variable defined to an empty value? Well, the answer is:

        % penv -S varname=''
    

And that is all - your environment is all set up!


4.3. Running programs using the defined settings

To run a program with an environment modified by the settings you have defined for a directory, simply run penv and tell it which program to execute:

    % penv program args..

Yes, it really is as simple as that :)

Keep in mind, though, that penv runs only once, in your current directory, and only uses the settings for this directory. If you want to use it ``recursively'', e.g. for building a FreeBSD port's dependencies with specific settings for each dependency, you may want to have a look at the compatibility section for some patches to FreeBSD's make(1) command allowing it to invoke penv for each directory it recurses into.


4.4. Some examples of using penv

4.4.1. Building a FreeBSD port

The main reason penv was developed was the author's annoyance with having to specify the same variables over and over again when building FreeBSD ports. How does penv help with that? Well, judge for yourself. First, run the following sequence of commands just once:

    % cd /usr/ports/mail/vpopmail
    
    % penv -c mkdir
    
    % penv -S WITH_MYSQL=yes WITH_MYSQL_USER=vpopmail WITH_MYSQL_PASS=unguessable WITH_MYSQL_DB=mail
    
    % penv -S WITH_HARDQUOTA=n WITH_QMAIL_EXT=yes

Of course, the two penv -S invocations could have been done in a single command, but those are broken down for clarity.

Now, every time you want to rebuild the mail/vpopmail port, instead of typing all those variables on the make(1) command line, never really certain whether you have not forgotten a couple of them, all you need to do is:

    # cd /usr/ports/mail/vpopmail
    
    # penv make clean all install

..and penv will take care of passing all the necessary variables to the port build!


4.4.2. Building the FreeBSD base system from source

FreeBSD ports are not the only area when the same environment settings may have to be specified repeatedly. I, personally, like to build my FreeBSD base system with debug information, yet install it with stripped binaries to save space. So, instead of doing the following:

    % cd /usr/src
    
    % make -DNOCLEAN DEBUG_FLAGS=-g3 buildworld buildkernel
    
    % sudo make TMPDIR=/home/roam/tmp STRIP=-s installkernel installworld

..I have set things up as:

    % cd /usr/src
    
    % penv -c mkdir
    
    % penv -S NOCLEAN=yes DEBUG_FLAGS=-g3 TMPDIR=/home/roam/tmp STRIP=-s

..which allows me to later do simply:

    % cd /usr/src
    
    % penv make buildworld buildkernel && sudo penv make installkernel installworld

..and be done with it.


4.4.3. Building FreeBSD Doc Project style documentation

Another FreeBSD-related example :) The documentation for many Ringlet software projects uses the build infrastructure of The FreeBSD Documentation Project. Some projects require tweaking some environment variables to get things right. Again, specifying those variables on the make(1) command line at every invocation may be tiresome; specifying them in the global make.conf configuration file would not do, because they would conflict with the settings for other projects. And again, penv comes to the rescue! :)

    % cd /home/roam/doc/fbsd-bg
    
    % penv -c mkdir
    
    % penv -S DOCDIR=/home/roam/doc/fbsd-bg DOC_PREFIX_NAME=fbsd-bg

After doing this, building the documentation with the correct settings is a breeze :)


5. Compatibility and interoperability

5.1. Making FreeBSD's make(1) invoke penv automatically

As far as I know, there are no compatibility problems that might arise if using penv with other programs. However, there may be situations when the abilities of penv to examine and set environment settings may appear to be inadequate. A simple example is, again, the FreeBSD Ports Collection. There are some ports which ``depend'' on other ports for their build, installation or correct operation. If penv is used to define settings for e.g. the mail/qmail port, those settings will not be ``picked up'' when building the mail/vpopmail port, which has a dependency on mail/qmail. The reason is simple - penv make issued in the mail/vpopmail/ directory would only honor the settings in the /var/db/penv/mail/vpopmail/ environment directory, and completely ignore those placed in /var/db/penv/mail/qmail/. It is somewhat uncomfortable to install/update each port individually, so that make(1) builds it with the correct settings, instead of just building the ``highest-level'' port and having the ``lower-level'' ports be installed with the correct settings right away.

This is why I have developed a simple patch to the FreeBSD make(1) program. It examines the values of the MAKEENVPROC and MAKEENVDIR variables each time make(1) is invoked, and, if needed, invokes an environment processor. The processor's name and arguments are defined in the MAKEENVPROC variable; make(1) invokes it using /bin/sh, so beware of shell metacharacters and such. To avoid unnecessary overheard, the processor is only invoked if the current working directory matches a regular expression pattern defined in the MAKEENVDIR variable.

The patch may be obtained from http://people.FreeBSD.org/~roam/bsd/makeenvproc.patch. An example use would be to set MAKEENVPROC to penv -L and MAKEENVDIR to /ports/. This would make make(1) invoke penv -L automatically in each directory under /usr/ports/ or /home/roam/fbsd/ports/, yet not affect e.g. building the FreeBSD base system in /usr/src/. Of course, this may be further refined by setting MAKEENVDIR to e.g. /ports/|/usr/home/roam/(.*/)?(doc|www)(/|$); this would cause make(1) to invoke penv in any directory containing a ports path component, and in any directory under /usr/home/roam/ which either contains or ends in doc or www. A needlessly complicated example, maybe, but one that I use nevertheless :)


6. A brief look at the penv source tree

The following is a list of the more important files in the source tree of penv, along with a brief explanation of their purpose.

The directory which holds the whole source tree may be named differently depending on where or how the source tree was acquired. In a tree checked out of the Ringlet CVS repository, this directory will be named penv/. In a tree extracted from a tar/gzip archive, this directory will be named something like penv-1.2p1/.


This, and other documents, can be downloaded from http://devel.ringlet.net/.

For questions about this documentation, e-mail <roam@ringlet.net>.