The module cover
provides a set of functions for coverage
analysis of Erlang programs, counting how many times each
executable line is executed.
Coverage analysis can be used to verify test cases, making sure all relevant code is covered, and may be helpful when looking for bottlenecks in the code.
Assume that a test case for the following program should be verified:
-module(channel). -behaviour(gen_server). -export([start_link/0,stop/0]). -export([alloc/0,free/1]). % client interface -export([init/1,handle_call/3,terminate/2]). % callback functions start_link() -> gen_server:start_link({local,channel},channel,[],[]). stop() -> gen_server:call(channel,stop). %%%-Client interface functions------------------------------------------- alloc() -> gen_server:call(channel,alloc). free(Channel) -> gen_server:call(channel,{free,Channel}). %%%-gen_server callback functions---------------------------------------- init(_Arg) -> {ok,channels()}. handle_call(stop,Client,Channels) -> {stop,normal,ok,Channels}; handle_call(alloc,Client,Channels) -> {Ch,Channels2} = alloc(Channels), {reply,{ok,Ch},Channels2}; handle_call({free,Channel},Client,Channels) -> Channels2 = free(Channel,Channels), {reply,ok,Channels2}. terminate(_Reason,Channels) -> ok. %%%-Internal functions--------------------------------------------------- channels() -> [ch1,ch2,ch3]. alloc([Channel|Channels]) -> {Channel,Channels}; alloc([]) -> false. free(Channel,Channels) -> [Channel|Channels].
The test case is implemented as follows:
-module(test). -export([s/0]). s() -> {ok,Pid} = channel:start_link(), {ok,Ch1} = channel:alloc(), ok = channel:free(Ch1), ok = channel:stop().
First of all, Cover must be started. This spawns a process which owns the Cover database where all coverage data will be stored.
1> cover:start(). {ok,<0.30.0>}
Before any analysis can take place, the involved modules must be
Cover compiled. This means that some extra information is
added to the module before it is compiled into a binary which then
is loaded. The source file of
the module is not affected and no .beam
file is created.
2> cover:compile_module(channel). {ok,channel}
Each time a function in the Cover compiled module channel
is called, information about the call will be added to the Cover
database. Run the test case:
3> test:s(). ok
Cover analysis is performed by examining the contents of the Cover
database. The output is determined by two parameters, Level
and Analysis
. Analysis
is either coverage
or
calls
and determines the type of the analysis. Level
is either module
, function
, clause
, or
line
and determines the level of the analysis.
Analysis of type coverage
is used to find out how much of
the code has been executed and how much has not been executed.
Coverage is represented by a tuple {Cov,NotCov}
, where
Cov
is the number of executable lines that have been executed
at least once and NotCov
is the number of executable lines
that have not been executed.
If the analysis is made on module level, the result is given for
the entire module as a tuple {Module,{Cov,NotCov}}
:
4> cover:analyse(channel,coverage,module). {ok,{channel,{14,1}}}
For channel
, the result shows that 14 lines in the module
are covered but one line is not covered.
If the analysis is made on function level, the result is given as
a list of tuples {Function,{Cov,NotCov}}
, one for each
function in the module. A function is specified by its module name,
function name and arity:
5> cover:analyse(channel,coverage,function). {ok,[{{channel,start_link,0},{1,0}}, {{channel,stop,0},{1,0}}, {{channel,alloc,0},{1,0}}, {{channel,free,1},{1,0}}, {{channel,init,1},{1,0}}, {{channel,handle_call,3},{5,0}}, {{channel,terminate,2},{1,0}}, {{channel,channels,0},{1,0}}, {{channel,alloc,1},{1,1}}, {{channel,free,2},{1,0}}]}
For channel
, the result shows that the uncovered line is in
the function channel:alloc/1
.
If the analysis is made on clause level, the result is given as
a list of tuples {Clause,{Cov,NotCov}}
, one for each
function clause in the module. A clause is specified by its module
name, function name, arity and position within the function
definition:
6> cover:analyse(channel,coverage,clause). {ok,[{{channel,start_link,0,1},{1,0}}, {{channel,stop,0,1},{1,0}}, {{channel,alloc,0,1},{1,0}}, {{channel,free,1,1},{1,0}}, {{channel,init,1,1},{1,0}}, {{channel,handle_call,3,1},{1,0}}, {{channel,handle_call,3,2},{2,0}}, {{channel,handle_call,3,3},{2,0}}, {{channel,terminate,2,1},{1,0}}, {{channel,channels,0,1},{1,0}}, {{channel,alloc,1,1},{1,0}}, {{channel,alloc,1,2},{0,1}}, {{channel,free,2,1},{1,0}}]}
For channel
, the result shows that the uncovered line is in
the second clause of channel:alloc/1
.
Finally, if the analysis is made on line level, the result is given
as a list of tuples {Line,{Cov,NotCov}}
, one for each
executable line in the source code. A line is specified by its
module name and line number.
7> cover:analyse(channel,coverage,line). {ok,[{{channel,9},{1,0}}, {{channel,12},{1,0}}, {{channel,17},{1,0}}, {{channel,20},{1,0}}, {{channel,25},{1,0}}, {{channel,28},{1,0}}, {{channel,31},{1,0}}, {{channel,32},{1,0}}, {{channel,35},{1,0}}, {{channel,36},{1,0}}, {{channel,39},{1,0}}, {{channel,44},{1,0}}, {{channel,47},{1,0}}, {{channel,49},{0,1}}, {{channel,52},{1,0}}]}
For channel
, the result shows that the uncovered line is
line number 49.
Analysis of type calls
is used to find out how many times
something has been called and is represented by an integer
Calls
.
If the analysis is made on module level, the result is given as a
tuple {Module,Calls}
. Here Calls
is the total number
of calls to functions in the module:
8> cover:analyse(channel,calls,module). {ok,{channel,12}}
For channel
, the result shows that a total of twelve calls
have been made to functions in the module.
If the analysis is made on function level, the result is given as
a list of tuples {Function,Calls}
. Here Calls
is
the number of calls to each function:
9> cover:analyse(channel,calls,function). {ok,[{{channel,start_link,0},1}, {{channel,stop,0},1}, {{channel,alloc,0},1}, {{channel,free,1},1}, {{channel,init,1},1}, {{channel,handle_call,3},3}, {{channel,terminate,2},1}, {{channel,channels,0},1}, {{channel,alloc,1},1}, {{channel,free,2},1}]}
For channel
, the result shows that handle_call/3
is
the most called function in the module (three calls). All other
functions have been called once.
If the analysis is made on clause level, the result is given as
a list of tuples {Clause,Calls}
. Here Calls
is
the number of calls to each function clause:
10> cover:analyse(channel,calls,clause). {ok,[{{channel,start_link,0,1},1}, {{channel,stop,0,1},1}, {{channel,alloc,0,1},1}, {{channel,free,1,1},1}, {{channel,init,1,1},1}, {{channel,handle_call,3,1},1}, {{channel,handle_call,3,2},1}, {{channel,handle_call,3,3},1}, {{channel,terminate,2,1},1}, {{channel,channels,0,1},1}, {{channel,alloc,1,1},1}, {{channel,alloc,1,2},0}, {{channel,free,2,1},1}]}
For channel
, the result shows that all clauses have been
called once, except the second clause of channel:alloc/1
which has not been called at all.
Finally, if the analysis is made on line level, the result is given
as a list of tuples {Line,Calls}
. Here Calls
is
the number of times each line has been executed:
11> cover:analyse(channel,calls,line). {ok,[{{channel,9},1}, {{channel,12},1}, {{channel,17},1}, {{channel,20},1}, {{channel,25},1}, {{channel,28},1}, {{channel,31},1}, {{channel,32},1}, {{channel,35},1}, {{channel,36},1}, {{channel,39},1}, {{channel,44},1}, {{channel,47},1}, {{channel,49},0}, {{channel,52},1}]}
For channel
, the result shows that all lines have been
executed once, except line number 49 which has not been executed at
all.
A line level calls analysis of channel
can be written to
a file using cover:analysis_to_file/1
:
12> cover:analyse_to_file(channel). {ok,"channel.COVER.out"}
The function creates a copy of channel.erl
where it for
each executable line is specified how many times that line has been
executed. The output file is called channel.COVER.out
.
File generated from channel.erl by COVER 2001-05-21 at 11:16:38 **************************************************************************** | -module(channel). | -behaviour(gen_server). | | -export([start_link/0,stop/0]). | -export([alloc/0,free/1]). % client interface | -export([init/1,handle_call/3,terminate/2]). % callback functions | | start_link() -> 1..| gen_server:start_link({local,channel},channel,[],[]). | | stop() -> 1..| gen_server:call(channel,stop). | | %%%-Client interface functions------------------------------------ | | alloc() -> 1..| gen_server:call(channel,alloc). | | free(Channel) -> 1..| gen_server:call(channel,{free,Channel}). | | %%%-gen_server callback functions--------------------------------- | | init(_Arg) -> 1..| {ok,channels()}. | | handle_call(stop,Client,Channels) -> 1..| {stop,normal,ok,Channels}; | | handle_call(alloc,Client,Channels) -> 1..| {Ch,Channels2} = alloc(Channels), 1..| {reply,{ok,Ch},Channels2}; | | handle_call({free,Channel},Client,Channels) -> 1..| Channels2 = free(Channel,Channels), 1..| {reply,ok,Channels2}. | | terminate(_Reason,Channels) -> 1..| ok. | | %%%-Internal functions-------------------------------------------- | | channels() -> 1..| [ch1,ch2,ch3]. | | alloc([Channel|Channels]) -> 1..| {Channel,Channels}; | alloc([]) -> 0..| false. | | free(Channel,Channels) -> 1..| [Channel|Channels].
By looking at the results from the analyses, it can be deducted
that the test case does not cover the case when all channels are
allocated and test.erl
should be extended accordingly.
Incidentally, when the test case is corrected a bug in channel
should indeed be discovered.
When the Cover analysis is ready, Cover is stopped and all Cover
compiled modules are unloaded.
The code for channel
is now loaded as usual from a
.beam
file in the current path.
13> code:which(channel). cover_compiled 14> cover:stop(). ok 15> code:which(channel). "./channel.beam"
Execution of code in Cover compiled modules is slower and more memory consuming than for regularly compiled modules. As the Cover database contains information about each executable line in each Cover compiled module, performance decreases proportionally to the size and number of the Cover compiled modules.
Cover uses the concept of executable lines, which is lines
of code containing an executable expression such as a matching or
a function call. A blank line or a line containing a comment,
function head or pattern in a case
- or receive
statement is not executable.
In the example below, lines number 2,4,6,8 and 11 are executable lines:
1: is_loaded(Module,Compiled) -> 2: case get_file(Module,Compiled) of 3: {ok,File} -> 4: case code:which(Module) of 5: ?TAG -> 6: {loaded,File}; 7: _ -> 8: unloaded 9: end; 10: false -> 11: false 12: end.
When a module is Cover compiled, it is also loaded using the normal
code loading mechanism of Erlang. This means that if a Cover
compiled module is re-loaded during a Cover session, for example
using c(Module)
, it will no longer be Cover compiled.
Use cover:is_compiled/1
or code:which/1
to see if
a module is Cover compiled (and still loaded) or not.
When Cover is stopped, all Cover compiled modules are unloaded.
To ease the use of Cover there is a web based user interface to Cover. The web based user interface to Cover is designed to be started and used via WebTool. It is possible to Cover compile Erlang modules and to generate printable Cover and Call analyses via the web based user interface.
Configure WebTool to manage the web based user interface to Cover, see WebTool User's Guide for more information. Start WebTool and point a browser to the start page of WebTool. Currently the web based user interface to Cover is only compatible with Internet Explorer and Netscape Navigator 4.0 and higher.
Click on the link marked WebCover in the topmost frame of WebTool. The main frame of the browser will then show the web based user interface to Cover.
To Cover compile a module or all the modules in a given directory select Compile in the left frame. Write the filename or the directory to be Cover compiled into the text field labeled Module or Directory. If other compile options than the standard compile options is needed, write them in the field labeled Compile Options. Click on the button labeled Compile. The module(s) will then be Cover compiled
If the name of the directory or file to Cover compile is unknown, it is possible to list the Erlang files in a directory, and change the working directory for the node from the web based user interface. This is done in the right part of the page.
To generate Cover or Call analysis, Cover compile the file either from the web based user interface or from the command line. Execute the code and drag the mouse over the module name, in the list of modules in the left frame. A pop-up menu will appear, select the wanted action from the menu and the result of the analyse will show up in the right frame.
To narrow the information in the Cover and Call analysis select one of the radio buttons at the top of the page.
To view the source file of a Cover compiled module with additional cover information for each line, drag the mouse over the module name in the left frame and a pop-up menu appears. Select Source File in the popup-menu. The source with line level analysis information will then come up in the left frame.