Source/Object Separation and Variant Builds

Up to this point all of the makefiles we have seen put the object files in the same directory as the source files. This is usually the way makefiles are written, and it's certainly the simplest way to do things. However, suppose you have to compile your program on both a linux machine and a Solaris machine. The binaries from the two machines are incompatible, of course. Unlike the traditional make, makepp is smart enough to know that if the last compilation was on linux, and the current compilation is on Solaris, a recompilation of everything is necessary.

But this still leaves a problem: when you recompile on Solaris, you wipe out your linux binaries. Then when you switch back to linux, you have to recompile everything again, even though the source files that haven't changed.

A related problem is if you build your program with several different options. Suppose for example that you usually compile your program with optimization:

CFLAGS	:= -O2

%.o: %.c
	$(CC) $(CFLAGS) -c $(input) -o $(output)

my_program: *.o
	$(CC) $(inputs) -o $(output)

However, you discover a bug, and you want to enable debugging on all files, so you do change CFLAGS:

CFLAGS	:= -g -DMALLOC_DEBUG

Makepp realizes that the build commands have changed, and it needs to recompile everything. But again, recompiling with debugging enabled wipes out your old binaries, so if you want to turn optimization back on, everything must be recompiled again, even the files that haven't changed.

The obvious solution to these problems is to put the architecture-dependent or build-variant-dependent files in a separate subdirectory. There are two basic techniques for doing this: explicitly specifying an alternate directory, or using repositories.

Explicit specifications of alternate directories

You could rewrite the rules in your makefile to dump the objects into a different directory, like this:

ARCH	:= $(shell uname -m)	# ARCH becomes the output from the uname -m command.
CFLAGS	:= -O2
OBJDIR	:= $(ARCH)-optim

$(OBJDIR)/%.o: %.c
	$(CC) $(CFLAGS) -c $(input) -o $(output)

$(OBJDIR)/my_program: $(OBJDIR)/*.o
	$(CC) $(inputs) -o $(output)

Now when you run makepp, ARCH is automatically set to something different for each architecture, and all of the objects are placed in a different directory for each architecture, so they don't overwrite each other. If you want to recompile turning on debugging, then you would have to change both CFLAGS and OBJDIR.

One problem with this approach is that implicit loading of makefiles will no longer work. The only place that makepp knows to look for a makefile when it needs to build something is in the directory of the file it's trying to build. If this is a problem for you, then you can explicitly tell makepp where to look using the load_makefile statement.

Repositories

Repositories are a magical way of using a makefile that is written to put objects in the same directory, but having makepp automatically put the objects in a different directory. Suppose we start with the original makefile above (before we modified it to put the objects in a different directory), and we've been working on linux so our source directory is filled with linux binaries. When we want to recompile our code on solaris instead of linux, we use the following command instead of just typing makepp:

% mkdir solaris
% cd solaris
% makepp -R ..

What the -R option to makepp does in this case is to declare the directory .. (which is the original source directory) as a repository. A repository is just a way of getting makepp to trick all of the actions into believing that all files in one directory tree are actually located in a different directory tree in the file system. In the above example, makepp pretends that all the files in .. (and all subdirectories of ..) are actually in the current directory (and corresponding subdirectories).

More precisely, a repository is a place where makepp looks if it needs to find a file that doesn't exist in the current directory tree. If the file exists in the current directory tree, it is used; if it doesn't exist, but a file exists in the repository, makepp makes a temporary symbolic link from the file in the repository to the current directory. (A symbolic link is an alias for the original file. It's like a copy, except that trying to access the link actually accesses the original file.) The rule actions then act on the file in the current directory, but actually reference the files in the repository.

In this example, initially we start off with a blank new directory solaris. (It doesn't have to be blank, of course, and it won't be the second time you run makepp.) Makepp is run in this directory, and it sees that there is no makefile there. However, there is a makefile in the repository, so it links in the one from the repository, and reads it. The pattern rule in the makefile that converts .c files into .o files causes makepp to link all the .c files that it needs from the repository, and run the compilation command from the solaris subdirectory. Therefore the .o files are now placed into the solaris subdirectory, not in the top level directory. When the build command is finished, any files linked from the repository are deleted, so the solaris subdirectory will contain only the binary files for Solaris. Any .o files that exist in the repository are unmodified, so when you go back to your linux machine and rerun makepp, most of your program will not have to be recompiled.

Sometimes it might be more convenient to use a different form of the repository command. The above three shell commands could be entirely replaced by the following one command:

% makepp -R solaris=. -F solaris

What this does is to say that the files in the current directory are to be linked into the solaris subdirectory as necessary. (The solaris subdirectory will be created automatically if it does not exist.) Then the -F option causes makepp to cd to the solaris directory and execute the makefile there (which will be linked from the repository).

Using a repository does not have the same drawbacks as explicitly specifying an object directory; makefiles will be implicitly loaded as expected, since as far as makepp is concerned, the makefile actually is in the same directory as the target files. However, if your build involves not just one but several directory trees, using repositories can become quite complicated.

Repositories are just a way of pretending that things located at one place in the file system are actually in a different place for the duration of the build. This is a very powerful technique that can be used for more than just separating your sources and binaries. For more details, see the reference manual.


Tutorial index | Next (debugging makefiles) | Previous (functions and variables)
Last modified: Tue Dec 26 21:08:53 PST 2000