How makepp finds include files and other hidden dependencies
Makepp can guess additional dependencies or targets for certain commands that it knows something about. This is especially important for C/C++ compilation, where it is too error-prone to list manually all of the include files that a given source file depends on. By looking at the compilation command and the source files themselves, makepp is able to determine accurately which object files need to be rebuilt when some include file changes.
Makepp looks at the first word of your command line.
If it recognizes it, it uses the scanner corresponding to
that first word. Specifically, it isolates the first word and looks it
up in its table; if nothing is found, it strips off the directory
information and looks it up again. (This means that you can specify the
path to your compiler and makepp will still recognize it.) Currently,
makepp recognizes most C/C++ compiler names. It also recognizes
commands invoking sh
and libtool
; for these commands, it skips to
the first word in the command that's not an option and looks it up in
the table again.
If makepp thinks it's compiling a C/C++ program but can't find a scanner, it will give a warning message to let you know. This usually means that you buried your compiler command too deeply in the action for makepp to find it. For example, I have seen rules like this:
%.o: %.c @echo Compiling $< now @gcc -c $< $(CFLAGS) -o $@
The first word of the action here is echo
, for which therer is no
scanner, so makepp will not scan for include files in this case.
The C/C++ scanner is activated by a command beginning with a compile
program that makepp knows about. (I've included every compiler I've
ever heard of, but if I missed your compiler, you can tell makepp about
it by adding an entry to the %Makesubs::scanners
array (see
Makesubs.pm in the distribution). Also send me email at
holt-makepp@gholt.net so I can include it in subsequent releases.
It looks at the command for -Idir
options specifying the include path
or -Ldir
options specifying the link path. It then scans any source
files for #include
directives, and also looks at the command line to
see if there are any source files or libraries mentioned which are not
listed as dependencies. It recognizes these by their extension.
This scanner gives a warning message if files included with #include
"file.h"
are not found in the include path, or in the directory
containing the file which is #includ
ing, or in /usr/include. No
warning is given if a file included with #include <file.h>
is not found. makepp assumes it is in some system include directory
that the compiler knows about, and that files in system include
directories won't change.
In addition, files in /usr/include, /usr/local/include,
/usr/X11R6/include, and any other directory which is not writable are
not scanned to see what they include. Makepp assumes that these files
won't change. (If you're running as root, the writability test is
performed with the UID and GID of the directory you ran makepp from.
This is so compiling a program as an ordinary user and then doing
make install
as root won't cause extra directories to be scanned.)
This is a fairly simple-minded scanner. It will get confused if you do things like this:
#ifdef INCLUDE_THIS #include "this.h" #endif
because it doesn't know about preprocessor conditionals. This is
usually harmless; it might cause additional extra files to be labelled
as dependencies (occasionally causing unnecessary rebuilds), or else it
might cause makepp to warn that the include file was not found. You can
either ignore the warning messages, or put an empty file this.h
out
there to shut makepp up.
Libtool is a very clever compilation system that greatly simplifies
making shared libraries by hiding all the system-dependent details away
in a shell script. The only difficulty is that the library binary files
are not actually stored in the same directory as the output
file--libtool actuall creates a subdirectory, .libs
, which contains
the real files. This is ordinarily not a problem, but makepp has to
know where the real binaries are if it is to link them in from a
repository. At the moment, libtool libraries (.la
files) are not
linked in from repositories; they are always rebuilt if needed. Also,
makepp at the moment is not able to use the dependency information that
is stored inside the .la
file itself. This will hopefully change
soon.
Makepp recognizes the following command words and skips over them
appropriately in in its search for the correct scanner:
condor_compile
, echo
ignore_error
libtool
, noecho
purify
, sh
.
The :quickscan and :smartscan rule options, if applicable, affect the way that files are scanned.
In :quickscan mode (the default), all include directives are assumed active. This allows for very efficient scanning.
In :smartscan mode, an attempt is made to interpret macros and expressions so that inactive include directives are ignored. For example, the executable produced by compiling the following C program ought not to depend on foo.h:
#if 0
# include "foo.h"
#endif
int main() { return 0; }
In the future, :smartscan might become the default.
Writing your own scanner is somewhat involved, but it is possible. There are 3 ways to hook into this:
If scanner foo is specified in a rule option, and there is a subroutine
called parser_foo defined in the makefile's package, then that subroutine
must return an object of type ActionParser
.
If scanner foo is specified in a rule option, and there is a subroutine called scanner_foo defined in the makefile's package, then that subroutine must do all of the scanning work. This is for compatibility with older versions of makepp, and is deprecated.
If no scanner is specified in a rule option, then an object of the
ActionParser
base class is used.
That object determines the scanner based on the first word of each command.
The base name is sought as a key in %scanners hash in the makefile's package,
which is seeded with all of the default scanners.
This hash is also affected by the register_scanner statement.
The value from this hash lookup is a coderef that either does the scanning
work (which is deprecated), or returns an object of type CommandParser
.
In most cases, objects of type CommandParser
should instantiate at least
one object of type Scanner
.
The Scanner
base class takes care of the distinction between quickscan
and smartscan.
Note that the behavior of Scanner
can be markedly affected by this
distinction, but that should be transparent to the derived class if it
is well-formed.
New derived Scanner
classes ought to be tested in both modes.
If you want to specify a particular CommandParser
class for a rule, then
you can specify a scanner with a rule option, and arrange for the parser_*
subroutine to return an object of type ActionParser::Specific
.
For more details, refer to the respective class documentation.
For examples, see CommandParser::Gcc
and CommandParser::Vcs
.
If the all of the scanner's important side effects are effected through calls
to methods of the CommandParser
base class, then those side effects can
be cached in the build info file, so that they can be played back by a
subsequent invocation of makepp without doing all of the costly scanning work.
This can save quite a bit of time, especially in smartscan mode.
If the scanner has other important side effects, then it should call the
Rule
object's mark_scaninfo_uncacheable method.
Otherwise, the scanner info retrieved from the build info may be inaccurate,
causing the build result possibly to be incorrect.
This method is called automatically when a value from the %scanners hash does
not return an object of type CommandParser
, or when the scanner is specified
with a rule option and no parser_* subroutine is defined.
Cached scan info is invalidated using criteria similar to those used for determining when the target is out of date. Similarly, it can be retrieved from a repository using criteria similar to those used for determining when a target can be linked in from a repository.
You can force makepp to ignore the cached scanner info with the
--force-rescan
option.
This is useful when a broken scanner may have caused incorrect scanner info
to be cached.