Subsections


F. Compiling the compiler

1 Introduction

The Free Pascal team releases at intervals a completely prepared package, with compiler and units all ready to use, the so-called releases. After a release, work on the compiler continues, bugs are fixed and features are added. The Free Pascal team doesn't make a new release whenever they change something in the compiler, instead the sources are available for anyone to use and compile. Compiled versions of RTL and compiler are also made daily, and put on the web.

There are, nevertheless, circumstances when the compiler must be recompiled manually. When changes are made to compiler code, or when the compiler is downloaded through CVS.

There are essentially 2 ways of recompiling the compiler: by hand, or using the makefiles. Each of these methods will be discussed.

2 Before starting

To compile the compiler easily, it is best to keep the following directory structure (a base directory of /pp/src is supposed, but that may be different):

/pp/src/Makefile
       /makefile.fpc
       /rtl/linux
           /inc
           /i386
           /...
       /compiler
When the makefiles should be used, the above directory tree must be used.

The compiler and rtl source are zipped in such a way that when both are unzipped in the same directory (/pp/src in the above) the above directory tree results.

There are 2 ways to start compiling the compiler and RTL. Both ways must be used, depending on the situation. Usually, the RTL must be compiled first, before compiling the compiler, after which the compiler is compiled using the current compiler. In some special cases the compiler must be compiled first, with a previously compiled RTL.

How to decide which should be compiled first? In general, the answer is that the RTL should be compiled first. There are 2 exceptions to this rule:

  1. The first case is when some of the internal routines in the RTL have changed, or if new internal routines appeared. Since the OLD compiler doesn't know about these changed internal routines, it will emit function calls that are based on the old compiled RTL, and hence are not correct. Either the result will not link, or the binary will give errors.
  2. The second case is when something is added to the RTL that the compiler needs to know about (a new default assembler mechanism, for example).
How to know if one of these things has occurred? There is no way to know, except by mailing the Free Pascal team. When the compiler cannot be recompiled when first compiling the RTL, then try the other way.

3 Compiling using make

When compiling with make it is necessary to have the above directory structure. Compiling the compiler is achieved with the target cycle.

Under normal circumstances, recompiling the compiler is limited to the following instructions (assuming you start in directory /pp/src):

cd compiler
make cycle
This will work only if the makefile is installed correctly and if the needed tools are present in the PATH. Which tools must be installed can be found in appendix [*].

The above instructions will do the following:

  1. Using the current compiler, the RTL is compiled in the correct directory, which is determined by the OS. e.g. under LINUX, the RTL is compiled in directory rtl/linux.
  2. The compiler is compiled using the newly compiled RTL. If successful, the newly compiled compiler executable is copied to a temporary executable.
  3. Using the temporary executable from the previous step, the RTL is re-compiled.
  4. Using the temporary executable and the newly compiled RTL from the last step, the compiler is compiled again.
The last two steps are repeated 3 times, until three passes have been made or until the generated compiler binary is equal to the binary it was compiled with. This process ensures that the compiler binary is correct.

Compiling for another target: When compiling the compiler for another target, it is necessary to specify the OS_TARGET makefile variable. It can be set to the following values: win32, go32v2, os2 and linux. As an example, cross-compilation for the go32v2 target from the win32 target is chosen:

cd compiler
make cycle OS_TARGET=go32v2
This will compile the go32v2 RTL, and compile a go32v2 compiler.

When compiling a new compiler and the compiler should be compiled using an existing compiled RTL, the all target must be used, and another RTL directory than the default (which is the ../rtl/$(OS_TARGET) directory) must be indicated. For instance, assuming that the compiled RTL units are in /pp/rtl, typing

cd compiler
make clean
make all UNITDIR=/pp/rtl
should use the RTL from the /pp/rtl dirrectory.

This will then compile the compiler using the RTL units in /pp/rtl. After this has been done, the 'make cycle' can be used, starting with this compiler:

make cycle PP=./ppc386
This will do the make cycle from above, but will start with the compiler that was generated by the make all instruction.

In all cases, many options can be passed to make to influence the compile process. In general, the makefiles add any needed compiler options to the command-line, so that the RTL and compiler can be compiled. Additional options (e.g. optimization options) can be specified by passing them in OPT.

4 Compiling by hand

Compiling by hand is difficult and tedious, but can be done. The compilation of RTL and compiler will be treated separately.

1 Compiling the RTL

To recompile the RTL, so a new compiler can be built, at least the following units must be built, in the order specified:
loaders
the program stubs, that are the startup code for each pascal program. These files have the .as extension, because they are written in assembler. They must be assembled with the GNU as assembler. These stubs are in the OS-dependent directory, except for LINUX, where they are in a processor dependent subdirectory of the LINUX directory (i386 or m68k).
system
the system unit. This unit is named differently on different systems: This unit resides in the OS-dependent subdirectories of the RTL.
strings
The strings unit. This unit resides in the inc subdirectory of the RTL.
dos
The dos unit. It resides in the OS-dependent subdirectory of the RTL. Possibly other units will be compiled as a consequence of trying to compile this unit (e.g. on LINUX, the linux unit will be compiled, on go32, the go32 unit will be compiled).
objects
the objects unit. It resides in the inc subdirectory of the RTL.
To compile these units on a i386, the following statements will do:
ppc386 -Tlinux -b- -Fi../inc -Fi../i386 -FE. -di386 -Us -Sg syslinux.pp
ppc386 -Tlinux -b- -Fi../inc -Fi../i386 -FE. -di386 ../inc/strings.pp
ppc386 -Tlinux -b- -Fi../inc -Fi../i386 -FE. -di386 dos.pp
ppc386 -Tlinux -b- -Fi../inc -Fi../i386 -FE. -di386 ../inc/objects.pp
These are the minimum command-line options, needed to compile the RTL.

For another processor, the i386 should be changed into the appropriate processor. For another operating system (target) the syslinux should be changed in the appropriate system unit file, and the target OS setting (-T) must be set accordingly.

Depending on the target OS there are other units that can be compiled, but which are not strictly needed to recompile the compiler. The following units are available for all plaforms:

objpas
Needed for Delphi mode. Needs -S2 as an option. Resides in the objpas subdirectory.
sysutils
many utility functions, like in Delphi. Resides in the objpas directory, and needs -S2 to compile.
typinfo
functions to access RTTI information, like Delphi. Resides in the objpas directory.
math
math functions like in Delphi. Resides in the objpas directory.
mmx
extensions for MMX class Intel processors. Resides in in the i386 directory.
getopts
a GNU compatible getopts unit. resides in the inc directory.
heaptrc
to debug the heap. resides in the inc directory.

2 Compiling the compiler

Compiling the compiler can be done with one statement. It's always best to remove all units from the compiler directory first, so something like

rm *.ppu *.o
on LINUX, and on DOS
del *.ppu
del *.o
After this, the compiler can be compiled with the following command-line:
ppc386 -Tlinux  -Fu../rtl/linux -di386 -dGDB pp.pas
So, the minimum options are:
  1. The target OS. Can be skipped when compiling for the same target as the compiler which is being used.
  2. A path to an RTL. Can be skipped if a correct fpc.cfg configuration is on the system. If the compiler should be compiled with the RTL that was compiled first, this should be ../rtl/OS (replace the OS with the appropriate operating system subdirectory of the RTL).
  3. A define with the processor for which the compiler is compiled for. Required.
  4. -dGDB is not strictly needed, but is better to add since otherwise compiling with debug information will not be possible.
  5. -Sg is needed, some parts of the compiler use goto statements (to be specific: the scanner).
So the absolute minimal command line is
ppc386 -di386 -Sg pp.pas

Some other command-line options can be used, but the above are the minimum. A list of recognised options can be found in table (FPCdefines) .


Table: Possible defines when compiling FPC
Define does what
USE_RHIDE Generates errors and warnings in a format recognized
  by rhide.
TP Needed to compile the compiler with Turbo or Borland Pascal.
Delphi Needed to compile the compiler with Delphi from Borland.
GDB Support of the GNU Debugger.
I386 Generate a compiler for the Intel i386+ processor family.
M68K Generate a compiler for the M68000 processor family.
USEOVERLAY Compiles a TP version which uses overlays.
EXTDEBUG Some extra debug code is executed.
SUPPORT_MMX only i386: enables the compiler switch MMX which
  allows the compiler to generate MMX instructions.
EXTERN_MSG Don't compile the msgfiles in the compiler, always use
  external messagefiles (default for TP).
NOAG386INT no Intel Assembler output.
NOAG386NSM no NASM output.
NOAG386BIN leaves out the binary writer.

This list may be subject to change, the source file pp.pas always contains an up-to-date list.




2002-04-25