Applications are used for "packaging" system components. An application consists of a set of resources, such as modules, registered names, and processes.
By structuring the system into a well-defined set of applications, the system designer is forced to think about the system in terms of its sub-components and to decide which functionality each application should have.
Several operations are possible with an application. In particular, we can load, unload, start and stop an application.
When an application is loaded, the system checks that all the resources the application will need are present. If loading is successful, the application can be started at a later time.
When an application is started, an application_master
process is created. Application master is the
group leader of all the processes in the application.
The application master is aware of every process in the application. Thus, if the application was stopped the application master would terminate all processes, for which this application is responsible, in a controlled fashion.
When an application is unloaded, all code relating to the application is removed.
All operations on applications are coordinated by a single
process with the name application_controller
. The shaded
circles in the above diagram represent applications. Each
individual application is controlled by an application
master.
The resources required by each application are defined in an
application resource file. These files have the extension
.app
, which specifies the resources required by the
application and how the application should be started.
A resource file (ApplName.app
) has the following syntax:
{application, ApplName, [{description, String}, {vsn, String}, {id, String}, {modules, [{Mod1,Vsn1}, Mod2, {Mod3,Vsn3} .., {ModN,VsnN}]}, {maxP, Int | infinity}, {maxT, Seconds | infinity}, {registered, [Name1, Name2, ...]}, {applications, [Appl1, Appl2, .., ApplN]}, {included_applications, [Appl1, Appl2, .., ApplN]}, {env, [{Par1, Val1}, {Par2, Val2} .., {ParN, ValN}]}, {mod, {Mod, StartArgs}}, {start_phases, [{Phase, PhaseArgs}]}]}.
The keys have the following meanings:
description
vsn
id
modules
maxP
infinity
). The key maxP
is optional and defaults
to infinity
.
maxT
infinity
). The key maxT
is
optional and defaults to infinity
.
registered
applications
kernel
and stdlib
applications.
included_applications
application_controller
. Processes implemented in an
included application should be placed underneath a supervisor in
the including application. This key is optional and defaults to
an empty list.
env
ParX
is an atom, and the
associated ValX
can be any term. The env
key is
optional and defaults to an empty list.
mod
Mod:start(Type,
StartArgs)
, refer to the kernel reference manual.
When the application has stopped, by command or
because it terminates, the application master calls
Mod:stop(State)
to let the application clean up. If no
State
was returned from Mod:start/2
, then
Mod:stop([])
is called.
The mod
key should be omitted for applications which
are code libraries, such as the stdlib
application. These applications have no dynamic behavior of
their own and should not have a start function.start_phases
Mod:start_phase(Phase, Type,PhaseArgs)
, for each defined start phase.
(See also the Kernel reference manual).
Each parameter Phase
is an atom, and the
associated PhaseArgs
is a list of any terms.
The key start_phases
is optional, but the behavior of
the system is dependent on the key being defined or not (see kernel
reference manual and the Starting Applications
chapter below).
Each application should be placed in a separate directory.
This directory should be divided into several
sub-directories. The following sub-directories should exist in
the application directory .../ApplName
:
src
ebin
priv
include
This structure is necessary because tools such as
systools
rely on this structure to be able to generate
boot scripts and release packages. The release handling
procedure on target machines also depends on this structure
to be able to upgrade to new releases.
![]() |
In the target environment, the application directory name contains the application version. The following description of the sub-directories listed also indicates if the sub-directory is mandatory or optional.
|
Application specific configuration parameters should be
specified in the .env
key in the resource file. The application can then
call application:get_env(ApplName, Parameter)
to retrieve
the values for the configuration parameters. You can also override configuration parameters
by using the system configuration file. This file is
specified with the command line parameter -config
FileName
. The file FileName.config
contains a list of
configuration parameters for the applications. For example:
[{sasl, [{sasl_error_logger, tty}, {errlog_type, error}, {disk_space_check_interval, 10}]}, ... {ApplNameN, [{Par1, Val1}, {Par2, Val2}]}].
The parameters over ride can also be executed from the command line:
erl -ApplName Par1 Val1 Par2 Val2 ...
![]() |
Each term should be an Erlang term. However, in the Unix
shell, the term must be enclosed in single quotation marks. For
example: |
Each individual application is controlled by an application master.
The application master is responsible for all processes running in an application. The application master can kill all processes which belong to the application.
The application master assumes that the process started by the start function is responsible for the internal details of the application. This process is assigned the following special role:
An application can be started in one of three modes: permanent, transient, or temporary. Default value is temporary. The mode specifies what happens if the application dies.
An application can include other applications. Processes implemented in an included application should be placed underneath a supervisor in the including application. This way, you include applications in order to group them together.
An application which is not included by any other application is
called a primary application. An application can only be included by
one other application. If you want several applications to include an
application, it has to be designed as a library application
without a start function (the mod
key in the .app file).
When an application is loaded, the
application controller ensures that all included
applications are also loaded. When an application is started , the
application controller will only start the primary application by
default (see chapter, Starting Applications
).
![]() |
The information in the following paragraph only applies to the Erlang/OTP environment. |
When building a release, the included_applications
key,
which is specified in the application resource file, can be
overridden with the .rel
file. Therefore, all applications
have the implicit environment (env
) variable
included_applications
in order to read the current
configuration. This is useful when an application has multiple start
parts (included applications) depending on customer requirements, and where
there is no need install or load excluded parts in the
system. Refer to the section Release Structure in the System
Architecture Support Libraries Manual for more details.
In a distributed system with several cooperating Erlang nodes,
there is a need to control certain applications in a distributed
manner. Some applications may be specified to run on one of
several nodes. The application controllers on the different nodes
can be arranged to monitor each other. If one node goes down, the
application controller on another node notices this and restart
the application on its node. These applications are called
distributed applications, as opposed to local
applications, which are always started on the local node. An
example of a standard local application is kernel
; there is
always one local instance of kernel
running on an Erlang
node.
With this definition, a local application may be distributed in the sense that it uses services on other nodes, or cooperates with applications on other nodes. For example, the application controller sees the Mnesia DBMS started as a local application. In this description, only the control of applications is discussed.
Because a distributed application may move between nodes, some
addressing mechanism is required to ensure that it can be
addressed by other applications, regardless on which node it
currently executes. This issue is not addressed here, but the
standard Erlang modules global
or pg
can be used for
this purpose.
The configuration parameter distributed
for the
application kernel
defines which applications are
distributed, and on which nodes they may run. This parameter
is of the form {ApplName,[NodeDesc]}
or {ApplName,
Time, [NodeDesc]}
, where NodeDesc = Node |{Node, ...,
Node}
. This data structure specifies a list of nodes
where the application ApplName
may execute, and the
order in which these nodes should be used to start the
application. If the nodes are specified in a tuple, the order
is undefined. If a node crashes and Time
has been
specified, then the application controller will wait for
Time
milliseconds before attempting to restart the
application on another node. If Time
is not specified,
it defaults to 0, and if a node goes down the application is
restarted immediately on another node.
![]() |
All involved nodes have to have the same distribution
specification (nodes which an application may run on
and restart time) for all distributed applications
on all involved nodes. This is easiest accomplished
by using the same configuration parameter Distribution specifications can also be set by
|
For example, suppose that the application myapp
is
defined to be distributed as {myapp, 5000, [cp1@cave,
{cp2@cave, cp3@cave}]}
. Suppose further that all nodes are
up and running when myapp
is started. It is then
started at cp1
, as shown in the figure below.
If cp1
goes down, the system checks which one of the
other nodes, cp2
or cp3
, has the least number of
running applications, but waits for 5 seconds for cp1
to restart. If cp1
does not restart and cp2
runs
fewer applications than cp3,
then myapp
is
restarted on cp2
.
Suppose now that cp2
goes down as well and does not
restart within 5 seconds. myapp
is now restarted on
cp3
.
If cp2
now restarts, it will not restart myapp
,
because the order between nodes cp2
and cp3
is
not defined.
However, if cp1
restarts as well, the function
application:takeover/2
moves myapp
to
cp1
, because cp1
has a higher priority than
cp3
for this application. In this case,
Mod:start({takeover, cp3@cave}, StartArgs)
is executed
at cp1
.
For a distributed application, the Nodes
list
distribution specification must be the same at each involved
node. The application controllers check this when they contact
each other at start-up. If a mismatch is found, the
application controller, and thus the entire Erlang node,
terminates with reason {distribution_mismatch, ApplName,
Node}
. The reason for this behavior is that the
application may not start at all, it may start at several
nodes, or the application controller may be left hanging if there is a
mismatch in the specification.
The distributed
parameter can be changed at release upgrade.
The applications are however not automatically moved to the nodes which
have the (new) highest priority; this has to be made manually by the operator after
the release upgrade. New applications in the new distributed
parameter are ignored by dist_ac
. Such applications have
to be loaded (with distribution information) and started manually
after the release upgrade has been performed.
Each node which is included in the distribution specification
for a distributed application must make an
application:start
call for the distributed
application. The start-up sequence between local and distributed
applications is also synchronized.
To illustrate this point, suppose that a local application
named localapp
uses the distributed application
myapp
, described in the previous example, with the
distribution specification {myapp, 5000, [cp1@cave,
{cp2@cave, cp3@cave}]}
. It is assured that
localapp
will not be started on cp1
, cp2
or cp3
until
cp1
starts myapp
. In this way, the start of
distributed applications is a synchronization point for all
nodes that may run the application.
When a local application is dependent on a distributed application but is configured to run on a different node than the distributed application a synchronization problem occurs, this is solved by also starting the distributed application on the node where the local application will run. This ensures that the applications are started in the correct order on the right nodes.
Please note when the above configuration applies the local application must be loaded before the distributed application is started. Normally this is not a problem because all applications are loaded before any applications are started.
When distributed applications are specified, it is
required that the sync_nodes
functionality in the
kernel
application (see kernel(6)
) is used to
synchronize all involved nodes. If not used, the distribution
mechanism for distributed applications will not function
correctly.
A primary application can be started in one or two steps. The first step is mandatory and the purpose of it is to start the main supervisor of the application and possible permanent children. The second step is optional and its purpose is to synchronize processes within an application.
The start parameters for the first step are defined in the mod
key
in the resource file for the primary application.
The application master will evaluate the function
Mod:start(Type, StartArgs)
.
Mod
and StartArgs
parameters are defined in the mod
key.
The parameter Type
states what type of start is running, ie. normal, takeover
or fail-over.
Takeover signifies that the application is distributed and is to be moved from
another node to this node, due to operator invention or due to this (newly
started) node is defined as superior to the other node in the configuration
parameter distributed
.
Fail-over signifies that this node should start the application due to a crash of the node where the application was previously running.
All other starts will result in a normal start type.
Note: Fail-over is only valid if the start_phases
key is
defined, otherwise this start type is denoted as normal. If
older application versions without start phases are being used, it is possible
to set the
start_phases
key to an empty list.
The second step is executed only if the start_phases
key is defined in the
resource file of the primary application.
If a primary application has a defined key,
all its included applications must also have the key defined. There is, however, a
possibility to include applications without start phases by wrapping them in an
another application (how to do this is explained later).
In this step the application master
evaluates the function Mod:start_phase(Phase, Type, PhaseArgs)
for each
phase in the specified order. Mod
is defined in the mod
key,
Phase
and PhaseArgs
in the start_phases
key, and Type
is the same as in the first step. An example showing (a part of) an application's
resource file and the evaluated functions:
myApp.app : {mod, {myApp, StartArgs}, {included_applications, []}, {start_phases, [{init, InitArgs}, {go, GoArgs}]}, : evaluated functions myApp:start(Type, StartArgs) % start the main supervisor % and permanent children myApp:start_phase(init, Type, InitArgs) % start the application % in the init phase myApp:start_phase(go, Type, GoArgs) % start the application % in the go phase
A primary application is responsible for starting included applications.
The primary application can read the included applications start
phases by calling application:get_key(Application, start_phases)
. There is,
however, a possibility to automate the start of included applications,
but first an example where the synchronization is taken care by the primary application:
primApp.app : {mod, {primApp, PrimAppStartArgs}, {included_applications, [inclOne, inclTwo]}, {start_phases, [{init, InitArgs}, {go, GoArgs}]}, : inclOne.app : {mod, {inclOne, NotUsedArgs}, {start_phases, [{go, GoArgs1}]}, : inclTwo.app : {mod, {inclTwo, NotUsedArgs}, {start_phases, [{init, InitArgs2}, {go, GoArgs2}]}, : evaluated functions primApp:start(Type, PrimAppStartArgs) % start the main supervisor % and permanent children primApp:start_phase(init, Type, InitArgs) % start the PrimApp and InclTwo % applications in init phase primApp:start_phase(go, Type, GoArgs) % start all applications % in go phase
To automate the synchronization of the
included applications a predefined module application_starter
is to be used
as the Module in the mod
key. application_starter
takes the
Mod
and StartArgs
as parameters:
{mod, {application_starter, [{Mod, StartArgs}]}
.
It is possible to have unique start phases in different applications.
The order of the phases is defined in the primary applications
start_phases
key for the whole tree. This aserts that all
included applications must have their start phases defined in the
including applications' resource files.
The application_starter
will execute only one start phases at
a time, from left to right, searching and executing (where indicated)
start phases in the included applications.
The above example is continued below using an automatic start for included applications:
primApp.app : {mod, {application_starter, [primApp, PrimAppStartArgs]}, {included_applications, [inclOne, inclTwo]}, {start_phases, [{init, InitArgsPrim}, {go, GoArgsPrim}]}, : inclOne.app : {mod, {inclOne, NotUsedArgs}, {start_phases, [{go, GoArgs1}]}, : inclTwo.app : {mod, {inclTwo, NotUsedArgs}, {start_phases, [{init, InitArgs2}, {go, GoArgs2}]}, : evaluated functions primApp:start(Type, PrimAppStartArgs) % start the main supervisor % and permanent children primApp:start_phase(init, Type, InitArgsPrim) % start the primApp application % in the init phase inclTwo:start_phase(init, Type, InitArgs2) % start the inclTwo application % in the init phase primApp:start_phase(go, Type, GoArgsPrim) % start the primApp application % in the go phase inclOne:start_phase(go, Type, GoArgs1) % start the inclOne application % in the go phase inclTwo:start_phase(go, Type, GoArgs2) % start the inclTwo application % in the go phase
For the reason mentioned above, the subsequently included
applications on the same branch must have their
start phases defined in all the including applications
resource files (start_phases
key). The application_starter
will then run the start phases in a sequentially correct start order.
Note:When starting the application tree the start phase does not descend level by level but follows a branch of the tree (starting applications as it descends) before moving to the next branch.
There is an example of the recursive use of the application starter below and a diagram of the go start phase flow.
primApp.app : {mod, {application_starter, [primApp, PrimAppStartArgs]}, {included_applications, [inclOne, inclTwoPrim]}, {start_phases, [{prim, PrimArgs}, {init, InitArgs}, {some, SomeArgs}, {spec, SpecArgs}, {go, GoArgs}]}, : inclOne.app : {mod, {inclOne, NotUsedArgs}, {included_applications, []}, {start_phases, [{spec, SpecArgs}, {go, GoArgsOne}]}, : inclTwoPrim.app : {mod, {application_starter, [inclTwoPrim, NotUsedArgs]}, {included_applications, [incl2A, incl2B]}, {start_phases, [{init, []}, {some, []}, {go, []}]}, : incl2A.app : {mod, {incl2A, []}, {included_applications, []}, {start_phases, [{some, SomeArgs2A}, {go, GoArgs2A}]}, : incl2B.app : {mod, {incl2B, []}, {included_applications, []}, {start_phases, [{init, InitArgs2B}]}, : evaluated functions primApp:start(Type, PrimAppStartArgs) % start the main supervisor % and permanent children primApp:start_phase(prim, Type, PrimArgs) primApp:start_phase(init, Type, InitArgs) inclTwoPrim:start_phase(init, Type, []) incl2B:start_phase(init, Type, InitArgs2B) primApp:start_phase(some, Type, SomeArgs) inclTwoPrim:start_phase(some, Type, []) incl2A:start_phase(some, Type, SomeArgs2A) primApp:start_phase(spec, Type, SpecArgs) inclOne:start_phase(spec, Type, SpecArgs) primApp:start_phase(go, Type, GoArgs) inclOne:start_phase(go, Type, GoArgsOne) inclTwoPrim:start_phase(go, Type, []) incl2A:start_phase(go, Type, GoArgs2A)
Mixing applications with start_phases
keys together with applications
which have no start_phases
keys is not allowed.
The wrapper application's only function is to start an application which has
no start_phases
key. Without a wrapper the application will not start.
primApp.app : {mod, {primApp, PrimAppStartArgs}, {included_applications, [with, wrapper]}, {start_phases, [{init, InitArgs}, {go, GoArgs}]}, : with.app : {mod, {with, NotUsedArgs}, {included_applications, []}, {start_phases, [{init, InitArgsW}, {go, GoArgsW}]}, : wrapper.app : {mod, {wrapper, NotUsedArgs}, {included_applications, []}, {start_phases, [{init, WrapArgs}]}, : evaluated functions primApp:start(Type, PrimAppStartArgs) % start the main supervisor % and permanent children primApp:start_phase(init, Type, InitArgs) % start the primary application % in the init phase with:start_phase(init, Type, InitArgsW) % start the with application % in init phase wrapper:start_phase(init, Type, WrapArgs) % start the without % application primApp:start_phase(go, Type, GoArgs) % start the primary application % in the go phase with:start_phase(go, Type, GoArgsW) % start the with application % in go phase
The following example assumes some familiarity with the Erlang
boot script
The example shows a simple boot script and config file for nodes
cp1
, cp2
, cp3
and cp4
, and
applications myapp
and localappl
. These are the
nodes and applications which are described in the previous
section, with the exception that node cp4
has been
added. This node only executes the local application
localapp
.
The following boot script is used at all four nodes. Each
node makes an application:start
call for myapp
,
although this distributed application is started only at one
node.
{script,{"Dist Test","1.0"}, [{preLoaded, [init,erl_prim_loader]}, {progress, preloaded}, <...> {progress,init_kernel_started}, {apply,{application,load, [{application, myapp, [{description,"MYAPP"}, {vsn,"1.0"}, {modules, []}, {applications,[kernel, stdlib]}, {env,[]}]}]}}, {apply,{application,load, [{application, localapp, [{description,"LOCALAPP"}, {vsn,"1.0"}, {modules, []}, {applications,[kernel, stdlib, myapp]}, {env,[]}]}]}}, {progress,applications_loaded}, {apply,{application,start,[kernel,permanent]}}, {apply,{application,start,[stdlib,permanent]}}, {apply,{application,start,[myapp,permanent]}}, {apply,{application,start,[localapp,permanent]}}, {progress,started}]}.
The following configuration file is used at all four nodes.
[{kernel, [{sync_nodes_optional, [cp1@cave, cp2@cave, cp3@cave, cp4@cave]}, {sync_nodes_timeout, 10000}, {distributed, [{myapp, [cp1@cave, {cp2@cave, cp3@cave}]}]}]}].
The application controller notices that cp4
cannot
start myapp
and then ensures that cp4
cannot start
localapp
until another node has started myapp
.