The CScout Refactoring Browser

Diomidis Spinellis
Department of Management Science and Technology
Athens University of Economics and Business
Athens, Greece
dds@aueb.gr

Introduction

CScout is a source code analyzer and refactoring browser for collections of C programs. It can process workspaces of multiple projects (we define a project as a collection of C source files that are linked together) mapping the complexity introduced by the C preprocessor back into the original C source code files. CScout takes advantage of modern hardware advances (fast processors and large memory capacities) to analyze C source code beyond the level of detail and accuracy provided by current compilers, linkers, and other source code analyzers. The analysis CScout performs takes into account the identifier scopes introduced by the C preprocessor and the C language proper scopes and namespaces.

CScout has already been applied on

CScout as a source code analyzer can:

More importantly, CScout helps you in refactoring code by identifying dead objects to remove, and automatically performing accurate global rename identifier refactorings, and various function argument refactorings. CScout will automatically rename identifiers

Furthermore, CScout allows you to refactor the arguments of functions and macros, introducing new arguments, deleting existing ones, or changing their order.

Walkthrough

If you are impatient, you can get an immediate feeling of CScout, by unpacking its distribution file, entering the example directory and typing You will then be able to use CScout and your browser to explore the source code of the one true awk (http://cm.bell-labs.com/who/bwk/index.html).

For a more structured walkthrough, read on. Consider the following C file, idtest.c

#define getval(x) ((x).val)

struct number {
        int id;
        double val;
} n;

struct character {
        int id;
        char val;
} c;

static int val;

main(int argc, char *argv[])
{
        int val;

        if (argc > 2)
                goto val;
        return getval(n) + getval(c);
        val: return 0;
}
Even though the file forms a contrived example, it will serve us to illustrate the basic concepts behind CScout's operation. Consider what would the correct renaming of one of the identifiers named val entail. CScout will help us to automate this process.

Although, we are dealing with a single file we need to specify its processing within the context of a workspace. In a realistic concept a workspace will specify how numerous projects consisting of multiple files will be processed; think of a workspace as a collection of Makefiles. CScout will operate across the many source files and related executables in the same way as it operates on our example file idtest.c.

A workspace specifies the set of files on which CScout will operate. Each workspace consists of a number of projects; a project is a set rules for linking together C files to form an executable. The workspace definition file is in our case very simple:

workspace example {
	project idtest {
		file idtest.c
	}
}
Our workspace, named example, consists of a single project, named idtest, that consists of a single C source file, idtest.c.

Our first step will be to transform the declarative workspace definition file into a processing script: a file with imperative processing directives that CScout will handle.

prompt> cswc example.csw >example.c
We then invoke CScout on the processing script (the compiled workspace definition file) example.c.
prompt> cscout example.c
Processing workspace example
Processing project idtest
Processing file idtest.c
Done processing file idtest.c
Done processing project idtest
Done processing workspace example
Post-processing our_path/example.c
Post-processing our_path/idtest.c
Processing identifiers
100%
We are now ready to serve you at http://localhost:8081
The output of CScout is quite verbose; when processing a large source code collection, the messages will serve to assure us that progress is being made.

The primary interface of CScout is Web-based, so once our files have been processed, we fire-up our Web browser and navigate to the CScout's URL. We leave the CScout process running; its job from now on will be to service the pages we request and perform the operations we specify.

Our browser will show us a page like the following:

Scout Main Page

Files

Functions and Macros

Identifiers

Operations

Main page - Web: Home Manual


CScout

In our first example we will only rename an identifier, but as is evident from the page's links CScout provides us with many powerfull tools.

By navigating through the links All files, idtest.c, and Source code with identifier hyperlinks we can see the source code with each recognised identifier marked as a hyperlink:

Source Code With Identifier Hyperlinks: your_path/idtest.c

(Use the tab key to move to each marked element.)


#define getval(x) ((x).val)

struct number {
        int id;
        double val;
n;

struct character {
        int id;
        char val;
c;

static int val;

main(int argc, char *argv[])
{
        int val;

        if (argc > 2)
                goto val;
        return getval(n) + getval(c);
        val: return 0;
}

Main page

Clicking on the first identifier val (in the macro definition) we are taken to a page specifying the identifier's details. There we can specify the identifier's new name, e.g. value.

Identifier: val

  • Read-only: No
  • Tag for struct/union/enum: No
  • Member of struct/union: Yes
  • Label: No
  • Ordinary identifier: No
  • Macro: No
  • Undefined macro: No
  • Macro argument: No
  • File scope: No
  • Project scope: No
  • Typedef: No
  • Crosses file boundary: No
  • Unused: No
  • Matches 3 occurence(s)
  • Appears in project(s):
    • idtest
  • Substitute with:

Dependent Files (Writable)

Dependent Files (All)

Main page

Clicking on the marked source hyperlink, CScout will show us again the corresponding source code, but with only the identifiers val matches marked as hyperlinks:

Identifier val: C:\dds\src\Research\cscout\refactor\idtest.c

(Use the tab key to move to each marked element.)


#define getval(x) ((x).val)

struct number {
        int id;
        double val;
} n;

struct character {
        int id;
        char val;
} c;

static int val;

main(int argc, char *argv[])
{
        int val;

        if (argc > 2)
                goto val;
        return getval(n) + getval(c);
        val: return 0;
}

Main page

The marked identifiers will be all the ones and the only ones the replacement we specified will affect. Similarly we can specify the replacement of the val label, the static variable, or the local variable; each one will only affect the relevant identifiers.

Selecting the hyperlink Exit - saving changes from the CScout's main page will commit our changes, modifying the file idtest.c.

Installation and Setup

System Requirements

To run CScout your system must satisfy the following requirements:

Installation and Configuration

From this point onward we use the term Unix to refer to Unix-like systems like GNU/Linux and FreeBSD, and Windows to refer to Microsoft Windows systems.

You install CScout in eight steps:

  1. Unpack the distribution file on your system.

  2. Copy the executable files cscout, cswc and csmake (under Unix) or cscout.exe, cswc.bat and csmake.bat (under Windows) from the bin directory into a directory that is part of your path. Under Unix /usr/local/bin is a common suitable choice. Under Windows C:\WINNT\system32 is a location you could use, if your system is not better organized.

  3. Under Windows adjust the second line of the file cswc.bat and csmake.bat to point to the directory where you installed the corresponding file.

  4. Copy the directory etc to the final installation place you prefer (renaming it, if you wish), and arrange for the environment variable CSCOUT_HOME to point to it. As an example, under Unix you would probably have the directory installed as /usr/local/etc/cscout. Under Unix, you can permanently set the CSCOUT_HOME environment variable by editing a file named .profile (sh and derivative shells) or .login (csh and derivative shells) in your home directory. Under Windows (NT, 2000, XP, and later editions), you can set environment variables through an option in: Control Panel - System - Advanced - Environment Variables, on Windows-95/98/Me you will need to edit the file c:\autoexec.bat.

    Alternativelly, the contents of the directory etc will be searched in $HOME/.cscout and the current directory's .cscout directory.

  5. Go in the CScout etc directory and copy the file pair cscout_incs.PLATFORM and cscout_defs.PLATFORM (where PLATFORM is the operating system and the compiler that most closely resemble your setup) as cscout_incs.h and cscout_defs.h.

    In most cases you want CScout to process your code using the include files of the compiler you are normally using. This will allow CScout to handle programs using the libraries and facilities available in your environment (e.g. Unix system calls or the Windows API). If your programs are written in ANSI C and do not use any additional include files, you can use the .GENERIC files and rely on the include files supplied with the CScout distribution.

  6. If you decided to use the .GENERIC files copy the include directory to an appropriate location (e.g. /usr/local/include/cscout under Unix).

  7. Edit the file cscout_incs.h to specify the location where your compiler's (or the generic) include files reside.

  8. If the compiler you are using does not match any of the files supplied, start with the .GENERIC file set and add suitable definitions to sidestep the problems caused by the extensions your compiler supports. As an example, if your compiler supports a quad_double type and associated keyword with semantics roughly equivalent to double you would add a line in cscout_incs:
    #define quad_double double
    
    Have a look in the existing cscout_defs files to see what might be required.
Note that there is nothing magical about the installation steps described above; feel free to follow them in whatever way matches your setup and environment, as long as you achieve the desired results.

Workflow

The following diagram illustrates the data flow when working with CScout.
Data flow in a CScout project
The thick-lined objects depict active processes; the thin-lined objects depict data. CScout will analyze and process C source code under the directions of a processing script. After some user interactions through a web browser CScout can write out the modified source code. The supported version of CScout can also convert the C source code into an SQL script that can be further analyzed and processesed through an RDBMS.

There are three ways to generate the processing script:

  1. Through a workspace definition file, processed by the workspace compiler cswc.
  2. By having the csmake command monitor the build process.
  3. By tailoring a project's build process to generate a processing script.
Each method has different advantages and disadvantages. Therefore, you should probably select the method that better suits your needs, and not bother with the others.

Workspace definition files offer by far the most readable and transparent way to setup a CScout workspace. They are declarative and express exactly the operations that CScout will perform. On the other hand, they can be difficult to specify for an existing large project and they must be kept in sync with the project's build process.

Running your make process under the csmake command is a very easy way to generate a CScout processing script. This method however only works if the essentials of your make process aren't too contrived. csmake can handle builds implemented through the Unix-related make, gcc, ld, ar, and mv commands. It has been successfuly tested on the Linux and FreeBSD kernels and the Apache web server. If csmake can deal with your project, you will be up and running in minutes; if not, you will only have lost those few minutes. Another advantage of the csmake method is that csmake will obtain from the compiler the predefined macros and the include file path. As a result you often don't have to tailor the files cscout_incs.h and cscout_defs.h to match you environment; you can directly use the supplied file cscout_defs.GENERIC_GCC.

Tailoring your project's build process to generate a CScout processing script is a final possibility. Here you gain maximum flexibility and integration with the project build system at the expense of having to modify the project's build procedure. If the project is relatively large and the build procedure is under your control, this may be an option worth investigating.

Defining Workspaces

A workspace definition provides CScout with instructions for parsing a set of C files; the task that is typically accomplished when compiling programs through the use of makefiles. CScout must always process all its source files in a single batch, so running it for each file from a makefile is not possible. Workspace definition files provide facilities for specifying linkage units (typically executable files - projects in the workspace definition file parlance) grouping together similar files and specifying include paths, read-only paths, and macros.

Workspace definition files are line-oriented and organized around C-like blocks. Comments are introduced using the # character. Consider the following simple example:

workspace echo {
	project echo {
		cd "/usr/src/bin/echo"
		file echo.c
	}
}
The above workspace definition consists of a single program (echo), which in turn consists of a single source file (echo.c).

See how we could expand this for two more programs, all residing in our system's /usr/src/bin directory:

workspace bin {
	cd "/usr/src/bin"
	ro_prefix "/usr/include"
	project cp {
		cd "cp"
		file cp.c utils.c
	}
	project echo {
		cd "echo"
		file echo.c
	}
	project date {
		cd "date"
		file date.c
	}
}
In the new bin workspace we have factored out the common source directory at the workspace level (cd "/usr/src/bin"), so that each project only specifies its directory relatively to the workspace directory (e.g. cd "date"). In addition, we have specified that files residing in the directory /usr/include are to be considered read-only (ro_prefix "/usr/include"). This is typically needed when the user running CScout has permission to modify the system's include files. Specifying one or more read-only prefixes allows CScout to distinguish between application identifiers and files, which you can modify, and system identifiers and files, which should not be changed.

The CScout workspace compiler cswc will read from its standard input, or from the file(s) specified on its command line, a workspace definition and produce on its standard output a processing script: a C-like file that CScout can process. You will have to redirect the cswc output to a file that will then get passed as an argument to CScout.

Workspace Definition Details

You can see the complete syntax of CScout workspaces in the following BNF grammar.
WORKSPACE:
	workspace NAME { WORKSPACE_ELEMENT ... }

WORKSPACE_ELEMENT:
	SCOPED_COMMAND
	GLOBAL_COMMAND
	cd "PATH"
	PROJECT

SCOPED_COMMAND:
	ipath "PATH"
	define MACRO
	define MACRO VALUE

GLOBAL_COMMAND:
	ro_prefix "PATH"
	readonly "FILENAME"

PROJECT:
	project NAME { PROJECT_ELEMENT ... }

PROJECT_ELEMENT:
	SCOPED_COMMAND
	cd "PATH"
	DIRECTORY
	FILE

DIRECTORY:
	directory PATH  { DIRECTORY_ELEMENT ... }

DIRECTORY_ELEMENT:
	SCOPED_COMMAND
	FILE

FILE:
	file FILENAME ...
	file "FILENAME" { FILESPEC ... }

FILESPEC:
	SCOPED_COMMAND
	cd "PATH"
	readonly
The above grammar essentially specifies that a workspace consists of projects, which consist of files or files in a directory. At the workspace level you can specify files and directories that are to be considered read-only using the readonly and ro_prefix commands. Both commands affect the complete workspace. The scoped commands (define and ipath) are used to specify macro definitions and the include path. Their scope is the block they appear in; when you exit the block (project, directory, or file) their definition is lost. You can therefore define a macro or an include path for the complete workspace, a specific project, files within a directory, or a single file. The syntax of the define command is the same as the one used in C programs. The cd command is also scoped; once you exit the block you return to the directory that was in effect in the outside block. Within a project you can either specify individual files using the file command, or express a grouping of files in a directory using the directory command. The directory command's name is the directory where a group of files resides and serves as an implicit cd command for the files it contains. Finally, files can be either specified directly as arguments to the file command, or file can be used to start a separate block. In the latter case the argument of file is the file name to process; the block can contain additional specifications (scoped commands or the readonly command without an argument) for processing that file.

The following workspace definition was used for processing the apache web server and includes most of the features and formulations we discussed.

workspace apache {
	cd "/usr/local/src/apache/src"

	ro_prefix "/usr/local/src/apache/src/include/ap_config"

	# Global project definitions
	define HTTPD_ROOT "/usr/local/apache"
	define SUEXEC_BIN "/usr/local/apache/bin/suexec"
	define SHARED_CORE_DIR "/usr/local/apache/libexec"
	define DEFAULT_PIDLOG "logs/httpd.pid"
	define DEFAULT_SCOREBOARD "logs/httpd.scoreboard"
	define DEFAULT_LOCKFILE "logs/httpd.lock"
	define DEFAULT_XFERLOG "logs/access_log"
	define DEFAULT_ERRORLOG "logs/error_log"
	define TYPES_CONFIG_FILE "conf/mime.types"
	define SERVER_CONFIG_FILE "conf/httpd.conf"
	define ACCESS_CONFIG_FILE "conf/access.conf"
	define RESOURCE_CONFIG_FILE "conf/srm.conf"

	define AUX_CFLAGS
	define LINUX 22 
	define USE_HSREGEX 
	define NO_DL_NEEDED

	# Give project-specific directory and include path properties
	project gen_uri_delims {
		cd "main"
		ipath "../os/unix"
		ipath "../include"
		file gen_uri_delims.c
	}

	# Alternative formulation; specify per-file properties
	project gen_test_char {
		file gen_test_char.c {
			cd "main"
			ipath "../os/unix"
			ipath "../include"
		}
	}

	# httpd executable; specify directory-based properties
	project httpd {
		directory main {
			ipath "../os/unix"
			ipath "../include"
 			file alloc.c buff.c http_config.c http_core.c
			file http_log.c http_main.c http_protocol.c
			file http_request.c http_vhost.c util.c util_date.c
			file util_script.c util_uri.c util_md5.c rfc1413.c
		}
		directory regex {
			ipath "."
			ipath "../os/unix"
			ipath "../include"
			define POSIX_MISTAKE
			file regcomp.c regexec.c regerror.c regfree.c
		}
		directory os/unix {
			ipath "../../os/unix"
			ipath "../../include"
			file os.c os-inline.c
		}
		directory ap {
			ipath "../os/unix"
			ipath "../include"
			file ap_cpystrn.c ap_execve.c ap_fnmatch.c ap_getpass.c 
			file ap_md5c.c ap_signal.c ap_slack.c ap_snprintf.c 
			file ap_sha1.c ap_checkpass.c ap_base64.c ap_ebcdic.c
		}
		directory modules/standard {
			ipath "../../os/unix"
			ipath "../../include"
			file mod_env.c mod_log_config.c mod_mime.c
			file mod_negotiation.c mod_status.c mod_include.c
			file mod_autoindex.c mod_dir.c mod_cgi.c mod_asis.c
			file mod_imap.c mod_actions.c mod_userdir.c
			file mod_alias.c mod_access.c mod_auth.c mod_setenvif.c
		}
		directory . {
			ipath "./os/unix"
			ipath "./include"
			file modules.c buildmark.c
		}
	}
}

Automated Generation of the Processing Script

In CScout from version 2.2 and onward you can you can also use the supplied tool csmake to directly generate CScout processing scripts by monitoring a project's make-based build process. For this to work your project's build must (probably) be based on a Unix or Unix-like system, and use make and gcc. The make process can also invoke ld, ar, and mv. Recursive make invocations among different directories are also supported.

The way to use csmake is fairly simple. You first arrange for performing a full build, for example by running

make clean
Then, instead of running make on the project's top-level directory you run csmake. When the build process has finished, csmake will leave in the directory where you started it a CScout processing script named make.cs.

csmake has been used out-of-the-box to run CScout on the Linux kernel version 2.6.11.4 and the Apache httpd version 2.2.3. It has also been used to process the FreeBSD 7-CURRENT kernel under its three Tier-1 architecture configurations by cross-compiling each configuration separately and merging the resulting CScout processing scripts. This is the shell script that did the job.

for a in amd64 i386 sparc64
do
        (
                cd sys/$a/conf/
                make LINT
                config LINT
        )
        export MAKEOBJDIRPREFIX=/home/dds/src/fbsd-head/obj/$a
        csmake buildkernel TARGET_ARCH=$a  KERNCONF=LINT
        mv make.cs make.$a.cs
done
cat make.*.cs >all.cs
sed -n 's/#pragma process "\(.*hack.c\)"/\1/p' all.cs | xargs touch
cscout all.cs

Tailoring the Build Process to Generate the Processing Script

It is relatively easy to integrate CScout into an existing IDE (such as Eclipse) or to provide an alternative method for specifying workspaces by directly creating a processing script from existing Makefiles. A processing script (what results from compiling a workspace file) is a C file containing a number of #pragma preprocessor directives. CScout uses the following pragmas:
#pragma echo "STRING"
Will display the STRING on CScout's standard output when that part of the file is reached.

Example:

#pragma echo "Processing workspace date\n"
#pragma ro_prefix "STRING"
Will add STRING to the list of filename prefixes that mark read-only files. This is a global setting.

Example:

#pragma ro_prefix "C:\gcc"
#pragma project "STRING"
Will set the current project to STRING. All identifiers and files processed from then on will belong to the given project.

Example:

#pragma project "date"
#pragma block_enter
Will enter a nested scope block. Two blocks are supported, the first block_enter will enter the project scope (linkage unit); the second encountered nested block_enter will enter the file scope (compilation unit).

#pragma block_exit
Will exit a nested scope block. The number of block_enter pragmas should match the number of block_exit pragmas and there should never be more than two block_enter pragmas in effect.

#pragma process "STRING"
Will analyze (CScout's equivalent to compiling) the C source file named STRING.

Example:

#pragma process "date.d"
#pragma pushd "STRING"
Will set the current directory to STRING, saving the previous current directory in a stack. From that point onward, all relative file accesses will start from the given directory.

Example:

#pragma pushd "cp"
#pragma popd
Will restore the current directory to the one in effect before a previously pushed directory. The number of pushd pragmas should match the number of popd pragmas.

#pragma includepath "STRING"
Will add STRING to the list of directories used for searching included files (the include path).

Example:

#pragma includepath "/usr/lib/gcc-lib/i386-redhat-linux/2.96/include"
#pragma clear_include
Will clear the include path, allowing the specification of a new one.

#pragma clear_defines
Will clear all defined macros allowing the specification of new ones. Should normally be executed before processing a new file. Note that macros can be defined using the normal #define C preprocessor directive.
The following is a complete example of a CScout processing script.
// workspace bin
#pragma echo "Processing workspace bin\n"
#pragma ro_prefix "/usr/include"
#pragma echo "Entering directory /usr/src/bin"
#pragma pushd "/usr/src/bin"
// project date
#pragma echo "Processing project date\n"
#pragma project "date"
#pragma block_enter
#pragma echo "Entering directory date"
#pragma pushd "date"
// file date.c
#pragma echo "Processing file date.c\n"
#pragma block_enter
#pragma clear_defines
#pragma clear_include
#include "/home/dds/src/cscout/cscout_defs.h"
#include "/home/dds/src/cscout/cscout_incs.h"
#pragma process "date.c"
#pragma block_exit
#pragma echo "Done processing file date.c\n"
#pragma echo "Exiting directory date\n"
#pragma popd
#pragma block_exit
#pragma echo "Done processing project date\n"
#pragma echo "Exiting directory /usr/src/bin\n"
#pragma popd
#pragma echo "Done processing workspace bin\n"

Case Study: Processing the FreeBSD Kernel (the Hard Way)

As a further example consider the steps for applying CScout on the FreeBSD kernel, before the existence of the csmake command.
  1. Configure a LINT or GENERIC version of each architecture's kernel.
    Example: config LINT
  2. Go to the compilation directory, update the dependencies (make depend) and compile (make). This step is used to create all automatically generated C and header files. Also during this step note the include path used, in order to provide CScout with the same specification.
  3. Remove the generated object files to force a make operation to rebuild them (rm *.o).
  4. Replace the C compiler invocation command in the Makefile with an appropriate series of shell commands.
    .include "$S/conf/kern.pre.mk"
    The code below was added after the line above
    NORMAL_C= echo '\#pragma echo "Processing file ${.IMPSRC}\n"' >>kernel.cs ;\
          echo '\#pragma block_enter' >>kernel.cs ;\
          echo '\#pragma clear_defines' >>kernel.cs ;\
          echo '\#pragma clear_include' >>kernel.cs ;\
          echo '\#include "cscout_defs.h"' >>kernel.cs ;\
          for i in $(INCLUDES) ; \
          do \
                  case $$i in \
                  -nostdinc) continue ;; \
                  -I-) continue ;; \
                  esac ; \
                  i=`echo $$i | sed 's/-I//'` ; \
                  echo '\#pragma includepath "'$$i'"' >>kernel.cs ; \
          done ; \
          echo '\#define _KERNEL 1' >>kernel.cs ;\
          echo '\#pragma process "opt_global.h"' >>kernel.cs ;\
          echo '\#pragma process "${.IMPSRC}"' >>kernel.cs ;\
          echo '\#pragma block_exit' >>kernel.cs ;\
          echo '\#pragma echo "Done processing file ${.IMPSRC}\n"' >>kernel.cs
    
  5. Create a cscout_incs.h file for each different architecture.
  6. Remove kernel.cs The existing file documents the way to do it.
  7. Run make on the custom Makefile
  8. Repeat for each different architecture
  9. Create a top-level CScout file to process all architectures:
    #pragma echo "Processing workspace FreeBSD kernel\n"
    
    #pragma echo "Entering directory sys/i386/compile/LINT\n"
    #pragma pushd "sys/i386/compile/LINT"
    #pragma echo "Processing project i386\n"
    #pragma project "i386"
    #pragma block_enter
    #include "kernel.cs"
    #pragma echo "Exiting directory sys/i386/compile/LINT\n"
    #pragma popd
    #pragma echo "Done processing project i386\n"
    #pragma block_exit
    
    #pragma echo "Entering directory sys/amd64/compile/GENERIC\n"
    // [...]
    // and so on for all architectures
    // [...]
    #pragma echo "Exiting directory sys/sparc64/compile/LINT\n"
    #pragma popd
    #pragma echo "Done processing project sparc64\n"
    #pragma block_exit
    
    Note that the block_enter and block_exit pragmas are furnished by this top-level file.
The run of the above specification (2 million unique lines) took 330 CPU minutes on a Rioworks HDAMA (AMD64) machine (2x1.8GHz Opteron 244 (in UP mode) - AMD 8111/8131 chipset, 8192MB mem) and required 1474MB of RAM. These are the complete metrics:

CScout Home

File Metrics

Writable Files
Number of files: 4310

File metricTotalMinMaxAvg
Number of characters625057700100834514502
Comment characters159217520850593694
Space characters79364010739681841
Number of line comments19040
Number of block comments1762530433740
Number of lines2063096027336478
Length of longest line3370490186778
Number of C strings13251901929630
Number of defined functions2958403336
Number of preprocessor directives26754202733662
Number of directly included files35408016088
Number of C statements67982504465157

CScout Execution

The CScout engine (cscout) requires as an argument a processing script, for example a cswc-compiled workspace definition file or the make.cs script generated by csmake. It will serially process each project and directory parsing the corresponding files specified in the workspace definition file, and then process once more each one of the files examined to establish the location of the identifiers. Note that the bulk of the work is performed in the first pass. During the first pass CScout may report warnings, errors, and fatal errors. Fatal errors will terminate processing, all other errors may result in an incorrect analysis of the particular code fragment. CScout only checks the code to the extend needed to perform its analysis; CScout will hapily process many illegal constructs.

The following lines illustrate the output of CScout when run on the bin workspace.

Entering directory /usr/src/bin
Processing project cp
Entering directory cp
Processing file cp.c
Done processing file cp.c
Processing file utils.c
Done processing file utils.c
Exiting directory cp
Done processing project cp
Processing project echo
Entering directory echo
Processing file echo.c
Done processing file echo.c
Exiting directory echo
Done processing project echo
Processing project date
Entering directory date
Processing file date.c
Done processing file date.c
Exiting directory date
Done processing project date
Exiting directory /usr/src/bin
Done processing workspace bin
Post-processing /home/dds/src/cscout/cscout_defs.h
Post-processing /home/dds/src/cscout/cscout_incs.h
Post-processing /usr/home/dds/src/cscout/bin.c
Post-processing /usr/include/ctype.h
Post-processing /usr/include/err.h
Post-processing /usr/include/errno.h
Post-processing /usr/include/fcntl.h
Post-processing /usr/include/fts.h
Post-processing /usr/include/limits.h
Post-processing /usr/include/locale.h
Post-processing /usr/include/machine/ansi.h
Post-processing /usr/include/machine/endian.h
Post-processing /usr/include/machine/limits.h
Post-processing /usr/include/machine/param.h
Post-processing /usr/include/machine/signal.h
Post-processing /usr/include/machine/trap.h
Post-processing /usr/include/machine/types.h
Post-processing /usr/include/machine/ucontext.h
Post-processing /usr/include/runetype.h
Post-processing /usr/include/stdio.h
Post-processing /usr/include/stdlib.h
Post-processing /usr/include/string.h
Post-processing /usr/include/sys/_posix.h
Post-processing /usr/include/sys/cdefs.h
Post-processing /usr/include/sys/inttypes.h
Post-processing /usr/include/sys/param.h
Post-processing /usr/include/sys/signal.h
Post-processing /usr/include/sys/stat.h
Post-processing /usr/include/sys/syslimits.h
Post-processing /usr/include/sys/time.h
Post-processing /usr/include/sys/types.h
Post-processing /usr/include/sys/ucontext.h
Post-processing /usr/include/sys/unistd.h
Post-processing /usr/include/sysexits.h
Post-processing /usr/include/syslog.h
Post-processing /usr/include/time.h
Post-processing /usr/include/unistd.h
Post-processing /vol/src/bin/cp/cp.c
Post-processing /vol/src/bin/cp/extern.h
Post-processing /vol/src/bin/cp/utils.c
Post-processing /vol/src/bin/date/date.c
Post-processing /vol/src/bin/date/extern.h
Post-processing /vol/src/bin/date/vary.h
Post-processing /vol/src/bin/echo/echo.c
Processing identifiers
100%
We are now ready to serve you at http://localhost:8081
After processing your files CScout will start operating as a Web server. At that point you must open a Web browser and connect to the location printed on its output. From that point onward your CScout contact is the Web browser interface; only fatal errors and progress indicators will appear on CScout's standard output. Depending on the version of CScout you have, you may also be able to perform some operations over the network. However, since CScout operates as a single-threaded process, you may experience delays when another user sends a complex query.

When CScout processes a large project it will contact our server over the Web to say hello. The unsupported version will also send us the identifier and file metrics of the project it has processed and register your CScout project for public browsing over the Web (the unsupported version is licensed only for use on free open-source software).

Preprocessor invocation

As an aid for configuring CScout for a different compiler you can run CScout and the workspace compiler with the optional -E command-line argument. The -E option will orchestrate both programs to act as a simple C preprocessor. The workspace definition file you should use in such a case should only specify a single file. The corresponding output of CScout will be the file with all preprocessor commands evaluated. If CScout reports an error in a place where a macro is invoked, you can examine the preprocessed output to see the result of the macro execution. During the CScout trials, this feature often located the use of nonstandard compiler extensions, that were hidden inside header files. To search for the corresponding error location in the postprocessed file use the name of a nearby identifier as a bookmark, since the line numbers will not match and CScout will not generate #line directives. Alternatively, you can rerun CScout on the preprocessed file.

Checking invocation

There are cases where you may only want to run CScout to see its error diagnostic messages. As an example, you may be running CScout as part of your daily build cycle to verify that the source code can always be parsed by CScout. The -c command-line option will cause CScout to immediately exit after processing the specified file.

The -c option is often used in conjunction with the -r option. The -r command-line option instructs CScout to report all superfluously included header files and identifiers that are either unused or wrongly scoped. Although it is easy to recognise when a header file must be included (if you do not follow the specification of the respective API, a compiler's error message will act as a reminder) detecting when an included header is no longer needed is a lot more difficult. Thus, as code changes, entire files are duplicated as source code templates, and functions are moved to different files, header files that were once needed may no longer be required. Their existence can confuse the programmers reading the code (why is this header file included?) and unnecessarily burden the compilation process. CScout can detect such files by keeping track of dependencies across files, and report included files that are not required. The following is an example of CScout's output:

$ cscout -rc awk.cs
Processing workspace awk
Processing project awk
Entering directory awk
Processing file awkgram.y
Done processing file awkgram.y
[...]
Processing file tran.c
Done processing file tran.c
Exiting directory awk
Done processing project awk
Done processing workspace awk
Post-processing /home/dds/src/cscout/example/.cscout/cscout_defs.h
[...]
Post-processing /home/dds/src/cscout/include/time.h
Processing identifiers
100%
/home/dds/src/cscout/example/awk/run.c:84: jexit: unused project scoped writable identifier
[...]
/home/dds/src/cscout/example/awk/awkgram.y:93: LASTTOKEN: unused file scoped writable identifier
/home/dds/src/cscout/example/awk/awk.h:152: CFREE: unused writable macro
[...]
/home/dds/src/cscout/example/awk/tran.c:44: CONVFMT: writable identifier should be made static
/home/dds/src/cscout/example/awk/lib.c:36: file: writable identifier should be made static
[...]
/home/dds/src/cscout/example/awk/lib.c:33: unused included file /home/dds/src/cscout/example/awk/ytab.h
/home/dds/src/cscout/example/awk/main.c:29: unused included file /home/dds/src/cscout/include/ctype.h
/home/dds/src/cscout/example/awk/main.c:35: unused included file /home/dds/src/cscout/example/awk/ytab.h
/home/dds/src/cscout/example/awk/tran.c:32: unused included file /home/dds/src/cscout/example/awk/ytab.h
Notice that there are two types of unused include files:
  1. Directly included files
  2. Included files that are only indirectly included
You will typically remove the #include directives for the directly included files. The files that are indirectly included and unused are a lot more tricky. They are brought into your file's compilation by the inclusion of another file. Even if you have control over the header file that included them and even if your file has no use for their contents, another file may require them, so in most cases it is best not to mess with those files. Finally note that it is possible to construct pathological examples of include files that CScout will not detect as being required. These will contain just parts of a statement or declaration that can not be related to the file including them (e.g. a single operator, or a comma):
/* Main file main.c */
main(int argc
#include "comma.h"
char *argv[])
{
}

/* File comma.h */
,
Although such a construct is legal C it is not used in practice.

Recently CScout processed a 190KLOC project that is under active development since 1989. The project consists of 231 files, containing 5249 include directives. Following CScout's analysis 765 include directives from 178 files were removed, without a single problem.

Basic Concepts

Before we begin our in-depth description of CScout's operation it is important to define the basic concepts we will encounter: identifiers, functions, and files. Although you may think you know what these elements stand for, in the CScout universe they have meanings sligthly different from what you may be used to.

Identifiers

A CScout identifier is the longest character sequence that can be correctly modified (e.g. renamed) in isolation. Identifiers that will have to be renamed in unison to obtain a correct program are grouped together and are treated as a single entity. Although you may think that, according to our definition, CScout identifiers are the same as C identifiers, this is the case only in the absence of the C preprocessor.

First of all, the preprocessor token concatenation feature can result in C identifiers that are composed of multiple CScout identifiers. Consider the following example, which uses a macro to define a number of different functions. (Yes, I am familiar with the C++ templates, this is just an example.)

#define typefun(name, type, op) \
type type ## _ ## name(type a, type b) { return a op b; }

typefun(add, int, +)
typefun(sub, int, -)
typefun(mul, int, *)
typefun(div, int, /)
typefun(add, double, +)
typefun(sub, double, -)
typefun(mul, double, *)
typefun(div, double, /)

main()
{
        printf("%d\n", int_add(5, 4));
        printf("%g\n", double_mul(3.14, 2.0));
}
In the CScout environment the int_add C identifier is actually composed of three separate parts:
  1. int
  2. _
  3. add
Renaming the int identifier into integer would change it in five different places: the argument to the four typefun macro invocations, and the part of int_add.

In addition, preprocessor macro definitions can confuse the notion of the C scope, bringing together scopes that would be considered separate in the context of the C language-proper. Consider the following (slightly contrived) example:

struct foo {
        int foo;
};

struct bar {
        int foo;
};

#define getpart(tag, name) (((struct tag *)p)->name)
#define getfoo(var) (var.foo)
#define get(name) (name(0) + ((struct name *)p)->name)
#define conditional(x) do {if (!x(0)) goto x; return x(0);} while(0)

int
foo(void *p)
{
        struct foo f;
        struct bar b;

foo:
        if (p && getpart(foo, foo))
                return getpart(bar, foo);
        else if (getfoo(f))
                return get(foo);
        else if (getfoo(b))
                conditional(foo);
        else
                return 0;
}
The identifier foo is occuring in a number of different scopes: Yet, the preprocessor macros and their use bring all the scopes together. If we decide to change one instance of the foo identifier, CScout will change all the instances marked below, in order to obtain a program that has the same meaning as the original one.

Identifier foo: test.c

(Use the tab key to move to each marked element.)


struct foo {
        int foo;
};

struct bar {
        int foo;
};

#define getpart(tag, name) (((struct tag *)p)->name)
#define getfoo(var) (var.foo)
#define get(name) (name(0) + ((struct name *)p)->name)
#define conditional(x) do {if (!x(0)) goto x; return x(0);} while(0)

int
foo(void *p)
{
        struct foo f;
        struct bar b;

foo:
        if (p && getpart(foofoo))
                return getpart(bar, foo);
        else if (getfoo(f))
                return get(foo);
        else if (getfoo(b))
                conditional(foo);
        else
                return 0;
}

Functions

CScout, with its integrated C preprocessor, considers as functions both the normal C functions and the function-like macros. It can therefore identify: The following example illustrates all the above cases.
#define macro() middlemacro()
#define middlemacro() innemacro()
#define innemacro() function1()
function1() {}
function2() {}
main() {
        macro();
        function2();
        function3();
        printf("Hello");
}
The corresponding call graph is as follows:

Calls from the main function

Note that in CScout functions are separate entities from identifiers. The name of a function can consist of multiple identifiers; an identifier can exist in more than one function names.

For instance, the page for the _ (underscore) identifier in the typefun macro example we saw earlier will appear as follows.

Identifier: _

Main page - Web: Home Manual


CScout 2.0 - 2004/07/31 12:37:12

Note how each function name is composed of three separate parts, and that this instance of the _ identifier occurs in 8 different function names.

Files

Given the complexities we discussed above, you may be pleased to know that in CScout files are more or less equivalent to the notion of file you are familiar with. The important thing to keep in mind is that CScout will consider all references to the same underlying file as equivalent, no matter how the file was named. Thus, different paths to the same file, or references to the same file via different symbolic links will end-up appearing as the same file in CScout.

One important feature of CScout concerning files has to do with the handling of files that are exact copies of each other. These may occur in the building of a large system for the sake of convenience; for example, one header file may be copied to various parts of the source code tree. CScout will locate identical files and group them together when reporting a file's details. Identifiers occuring in the same position of two identical files are considered equivalent; if you change the name of one of them the name of the other will also change. Moreover, when CScout reports unused identifiers it takes into account uses of an identifier from all instances of the identical files, not just one of them.

The Web Interface

The main screen CScout presents to your browser is divided into four sections: Most pages CScout sends to your browser are dynamically generated and may contain elements that can vary from one CScout invocation to the next. Therefore you should not bookmark source listings, or file or identifier detail pages, and expect them to be available on another CScout invocation. On the other hand, the pages containing results of identifier, function, or file queries can be freely bookmarked and are identified with a comment specifying the fact and a corresponding link.

File-spanning Writable Identifiers

Matching Identifiers

ADD
...
BTRUE
CALL

Elements 1 to 20 of 416.
Select page: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 next all
You can bookmark this link to save the respective query.

Main page

You can therefore use your browser's bookmark facility to ``store'' such queries for future use, or pass the URL around so that others can reproduce your results.

Also note that often a query's results are split into pages. The program's options allow you to specify how many elements you want to see on each page. Keep in mind that some browsers may choke on huge pages, so keep this number down to a reasonable number (say below 1000). You can navigate between result pages using the links at the bottom of each result page page. The link titled all will present all the query's results. It is most useful as a way to save all the query's results into a file, using a browser command like Save Link Target As ...

We will examine CScout's functionality using as an example the bin workspace we presented in the previous section.

File Elements

Although some of the file queries operate on identifier properties, all file queries produce file-list data as their result. Clicking on an element of a file list leads you to a page with a summary of the file.

File: /home/user/src/cscout/example/awk/main.c

Details

  • Read-only: No
  • Used in project(s):
    • awk
  • Other exact copies: (none)

Listings

Include Files

Metrics

MetricValue
Number of characters5155
Number of comment characters1878
Number of space characters754
Number of line comments0
Number of block comments27
Number of lines190
Maximum number of characters in a line107
Number of character strings25
Number of unprocessed lines0
Number of C preprocessor directives9
Number of processed C preprocessor conditionals (ifdef, if, elif)0
Number of defined C preprocessor function-like macros0
Number of defined C preprocessor object-like macros1
Number of preprocessed tokens893
Number of compiled tokens921
Number of copies of the file1
Number of statements123
Number of defined project-scope functions3
Number of defined file-scope (static) functions0
Number of defined project-scope variables13
Number of defined file-scope (static) variables0
Number of complete aggregate (struct/union) declarations0
Number of declared aggregate (struct/union) members0
Number of complete enumeration declarations0
Number of declared enumeration elements0
Number of directly included files8

Main page - Web: Home Manual


CScout
The page contains some representative metrics for the given file, the projects using this file, links for viewing the file's source code, and links for listing include file dependencies.

You can view a file's source code in five different forms:

  1. The plain source code, will only provide you the file's code text
  2. The source code with unprocessed regions marked, will enable you to see which parts of the file was not processed due to conditional compilation instructions. You may want to use the marked parts as a guide to construct a more inclusive workspace definition (perhaps by processing the project multiple times, with different preprocessor options).
      360 #if defined(__GNUC__) && defined(__STDC__)
      361 static __inline int __sputc(int _c, FILE *_p) {
      362         if (--_p->_w >= 0 || (_p->_w >= _p->_lbfsize && (char)_c != '\n'))
      363                 return (*_p->_p++ = _c);
      364         else
      365                 return (__swbuf(_c, _p));
      366 }
      367 #else
      368 /*
      369  * This has been tuned to generate reasonable code on the vax using pcc.
      370  */
      371 #define __sputc(c, p) \
      372         (--(p)->_w < 0 ? \
      373                 (p)->_w >= (p)->_lbfsize ? \
      374                         (*(p)->_p = (c)), *(p)->_p != '\n' ? \
      375                                 (int)*(p)->_p++ : \
      376                                 __swbuf('\n', p) : \
      377                         __swbuf((int)(c), p) : \
      378                 (*(p)->_p = (c), (int)*(p)->_p++))
      379 #endif
      380 
  3. Source code with identifier hyperlinks, will provide you with a page of the file's code text where each identifier is represented as a hyperlink leading to the identifier's page. The following is a representative example.
    int
    copy_fifo(from_statexists)
            struct stat *from_stat;
            int exists;
    {
            if (exists && unlink(to.p_path)) {
                    warn("unlink: %s", to.p_path);
                    return (1);
            }
            if (mkfifo(to.p_pathfrom_stat->st_mode)) {
                    warn("mkfifo: %s", to.p_path);
                    return (1);
            }
            return (pflag ? setfile(from_stat, 0) : 0);
    }

  4. As the above display can be overwhelming, you may prefer to browse the source code with hyperlinks only to project-global writable identifiers, which are typically the most important identifiers. Consider again how the above example would be displayed:
    int
    copy_fifo(from_stat, exists)
            struct stat *from_stat;
            int exists;
    {
            if (exists && unlink(to.p_path)) {
                    warn("unlink: %s", to.p_path);
                    return (1);
            }
            if (mkfifo(to.p_path, from_stat->st_mode)) {
                    warn("mkfifo: %s", to.p_path);
                    return (1);
            }
            return (pflag ? setfile(from_stat, 0) : 0);
    }
  5. Source code with hyperlinks to function and macro declarations provides you hyperlinks to the function pages for each function declaration (implicit or explict) and macro definition. Again, here is an example:
    #if !defined(_ANSI_SOURCE) && !defined(_POSIX_SOURCE)
    int     digittoint __P((int));
    int     isascii __P((int));
    int     isblank __P((int));
    int     ishexnumber __P((int));
    int     isideogram __P((int));
    int     isnumber __P((int));
    int     isphonogram __P((int));
    int     isrune __P((int));
    int     isspecial __P((int));
    int     toascii __P((int));
    #endif
    __END_DECLS

    #define __istype(c,f)    (!!__maskrune((c),(f)))

    #define isalnum(c)       __istype((c), _CTYPE_A|_CTYPE_D)
    #define isalpha(c)       __istype((c), _CTYPE_A)
    #define iscntrl(c)       __istype((c), _CTYPE_C)
    #define isdigit(c)       __isctype((c), _CTYPE_D) /* ANSI -- locale independent */
    #define isgraph(c)       __istype((c), _CTYPE_G)
    #define islower(c)       __istype((c), _CTYPE_L)
    #define isprint(c)       __istype((c), _CTYPE_R)
    #define ispunct(c)       __istype((c), _CTYPE_P)
    #define isspace(c)       __istype((c), _CTYPE_S)
    #define isupper(c)       __istype((c), _CTYPE_U)
    #define isxdigit(c)      __isctype((c), _CTYPE_X) /* ANSI -- locale independent */
    #define tolower(c)       __tolower(c)
    #define toupper(c)       __toupper(c)

File Metrics

File metrics produces a summary of the workspace's file-based metrics like the following:

File Metrics

Writable Files

Number of elements: 14

MetricTotalMinMaxAvg
Number of characters16062519234329711473.2
Number of comment characters30314063072165.29
Number of space characters2908129887352077.21
Number of line comments120120.857143
Number of block comments760019054.2857
Number of lines65571001913468.357
Maximum number of characters in a line11022410778.7143
Number of character strings742015453
Number of unprocessed lines12080.857143
Number of C preprocessor directives376010226.8571
Number of processed C preprocessor conditionals (ifdef, if, elif)6030.428571
Number of defined C preprocessor function-like macros300222.14286
Number of defined C preprocessor object-like macros16209211.5714
Number of preprocessed tokens39883354121892848.79
Number of compiled tokens441190140203151.36
Number of copies of the file14111
Number of statements429301589306.643
Number of defined project-scope functions16805112
Number of defined file-scope (static) functions2010.142857
Number of defined project-scope variables14903610.6429
Number of defined file-scope (static) variables191017413.6429
Number of complete aggregate (struct/union) declarations12060.857143
Number of declared aggregate (struct/union) members560324
Number of complete enumeration declarations0000
Number of declared enumeration elements0000
Number of directly included files820205.85714

Read-only Files

Number of elements: 14

MetricTotalMinMaxAvg
Number of characters3973722798762838.36
Number of comment characters2680510756951914.64
Number of space characters280513948200.357
Number of line comments0000
Number of block comments1282609.14286
Number of lines10741327576.7143
Maximum number of characters in a line1025508573.2143
Number of character strings4030.285714
Number of unprocessed lines17051.21429
Number of C preprocessor directives18615813.2857
Number of processed C preprocessor conditionals (ifdef, if, elif)27061.92857
Number of defined C preprocessor function-like macros300132.14286
Number of defined C preprocessor object-like macros860316.14286
Number of preprocessed tokens3107161068221.929
Number of compiled tokens17790602127.071
Number of copies of the file14111
Number of statements0000
Number of defined project-scope functions0000
Number of defined file-scope (static) functions0000
Number of defined project-scope variables6030.428571
Number of defined file-scope (static) variables0000
Number of complete aggregate (struct/union) declarations7030.5
Number of declared aggregate (struct/union) members560234
Number of complete enumeration declarations0000
Number of declared enumeration elements0000
Number of directly included files4010.285714

Main page - Web: Home Manual


CScout

All files

The "All files" link will list all the project's files, including source files, and directly and indirectly included files. You can use this list to create a "bill of materials" for the files your workspace requires to compile. The following is an example of the output:

All Files

You can bookmark this page to save the respective query

Main page


CScout 1.6 - 2003/06/04 15:14:51

Read-only files

The "Read-only files" link will typically show you the system files your project used. The following output was generated using the "Show file lists with file name in context" option.

Read-only Files

DirectoryFile
/usr/include/ ctype.h
/usr/include/ err.h
/usr/include/ errno.h
/usr/include/ fcntl.h
/usr/include/ fts.h
/usr/include/ limits.h
/usr/include/ locale.h
/usr/include/machine/ ansi.h
/usr/include/machine/ endian.h
/usr/include/machine/ limits.h
/usr/include/machine/ param.h
/usr/include/machine/ signal.h
/usr/include/machine/ trap.h
/usr/include/machine/ types.h
/usr/include/machine/ ucontext.h
/usr/include/ runetype.h
/usr/include/ stdio.h
/usr/include/ stdlib.h
/usr/include/ string.h
/usr/include/sys/ _posix.h
/usr/include/sys/ cdefs.h
/usr/include/sys/ inttypes.h
/usr/include/sys/ param.h
/usr/include/sys/ signal.h
/usr/include/sys/ stat.h
/usr/include/sys/ syslimits.h
/usr/include/sys/ time.h
/usr/include/sys/ types.h
/usr/include/sys/ ucontext.h
/usr/include/sys/ unistd.h
/usr/include/ sysexits.h
/usr/include/ syslog.h
/usr/include/ time.h
/usr/include/ unistd.h

You can bookmark this page to save the respective query

Main page - Web: Home Manual


CScout 2.0 - 2004/07/31 12:37:12

Writable files

Correspondingly the "Writable files" link will only show you all your workspace's source files:

Writable Files

You can bookmark this page to save the respective query

Main page


CScout 1.6 - 2003/06/04 15:14:51

Files containing unused project-scoped writable identifiers

The link ``files containing unused project-scoped writable identifiers'' performs an identifier query, but lists as output files containing matching identifiers. Specifically, the link will produce a list of files containing global (project-scoped) unused writable identifiers. Modern compilers can detect unused block-local or even file-local (static) identifiers, but detecting global identifiers is more tricky, since it requires processing of all files that will be linked together. The restriction to writable identifiers will filter-out noise generated through the use of the system's library functions.

In our example, the following list is generated:

Files Containing Unused Project-scoped Writable Identifiers

Matching Files

DirectoryFile
/vol/src/bin/cp/ cp.cmarked source
/vol/src/bin/date/ date.cmarked source

You can bookmark this page to save the respective query

Main page - Web: Home Manual


CScout 2.0 - 2004/07/31 12:37:12
The output contains the path to each file, and a link that will generate the file's source code with the offending identifiers marked as hyperlinks. You can use the ``marked source'' link to inspect the identifiers in the context of their source code; simply follow the link with your browser and press tab to go to each hyperlink. In our example the identifier will appear as follows:

void
setthetime(fmt, p, jflag, nflag)
        const char *fmt;
        register const char *p;
        int jflag, nflag;
{
        register struct tm *lt;
        struct timeval tv;
        const char *dot, *t;
        int century;

(In our case the function setthetime is declared as static, but not defined as such.)

Files containing unused file-scoped writable identifiers

The link ``files containing unused file-scoped writable identifiers'' performs an identifier query, but lists as output files containing matching identifiers. Specifically, the link will produce a list of files containing file-scoped (static) unused writable identifiers. Although some modern compilers can detect file-local identifiers, they fail to detect macros and some types of variable declarations. The CScout query is more general and can be more reliable. The restriction to writable identifiers will filter-out noise generated through the use of the system's library functions.

In our example, the following list is generated:

Files Containing Unused File-scoped Writable Identifiers

Matching Files

DirectoryFile
/vol/src/bin/cp/ cp.cmarked source
/vol/src/bin/cp/ utils.cmarked source
/vol/src/bin/date/ date.cmarked source
/vol/src/bin/echo/ echo.cmarked source

You can bookmark this page to save the respective query

Main page - Web: Home Manual


CScout 2.0 - 2004/07/31 12:37:12
In our case all identifiers located were the copyright and the rcsid identifiers.

#ifndef lint
static char const copyright[] =
"@(#) Copyright (c) 1989, 1993\n\
        The Regents of the University of California.  All rights reserved.\n";
#endif /* not lint */

#ifndef lint
#if 0
static char sccsid[] = "@(#)echo.c      8.1 (Berkeley) 5/31/93";
#endif
static const char rcsid[] =
  "$FreeBSD: src/bin/echo/echo.c,v 1.8.2.1 2001/08/01 02:33:32 obrien Exp $";
#endif /* not lint */

Later on we will explain how an identifier query could have used a regular expression to filter-out the noise generated by these two identifiers.

Writable .c files without any statements

The ``writable .c files without any statements'' will locate C files that do not contain any C statements. You can use it to locate files that only contain variable definitions, or files that are #ifdef'd out.

In our example, the result set only contains the processing script (the compiled workspace definition file).

Writable .c Files Without Any Statments

You can bookmark this page to save the respective query

Main page


CScout 1.6 - 2003/06/04 15:14:51
The processing script (the compiled workspace definition file) follows the C syntax, but only contains preprocessor directives (mostly CScout-specific #pragma commands) to drive the CScout's source code analysis.

Writable files containing unprocessed lines

The ``writable files containing unprocessed lines'' link will present you C files containing lines that were skipped by the C preprocossor, due to conditional compilation directives. The files are ordered according to the number of unprocessed lines (files with the largest number will appear on the top).

In our case the results are:

Writable Files Containing Unprocessed Lines

DirectoryFileNumber of unprocessed lines
/vol/src/bin/cp/ utils.c30
/home/dds/src/cscout/ cscout_defs.h3
/vol/src/bin/echo/ echo.c2
/vol/src/bin/date/ date.c2
/vol/src/bin/cp/ cp.c2
/home/dds/src/cscout/ cscout_incs.h2

You can bookmark this page to save the respective query

Main page - Web: Home Manual


CScout 2.0 - 2004/07/31 12:37:12
Lines skipped by the C preprocessor can be detrimental to the analysis and the refactoring you perform. If those lines contain live code that will be used under some other circumstances (a different platform, or different configuration options), then any results you obtain may miss important data. The list of files allows you to see if there are any large chunks of code that CScout ignored. If there are, think about specifying additional configuration options as preprocessor variables. If some configuration options are mutually exclusive you can process the same source multiple times, with different preprocessor variables set.

Writable files containing strings

The ``writable files containing strings'' link will present you C files containing C strings. In some applications user-messages are not supposed to be put in the source code, to aid localization efforts. This file query can then help you locate files that contain strings.

In our case the results are:

Writable Files Containing Strings

You can bookmark this page to save the respective query

Main page


CScout 1.6 - 2003/06/04 15:14:51

Writable .h files with #include directives

Some coding conventions dictate against recursive #include invocations. This query can be used to find files that break such a guideline. As usual, read-only system files are excluded; these typically use recursive #include invocations as a matter of course.

In our example, the result is:

Writable .h Files With #include directives

You can bookmark this page to save the respective query

Main page


CScout 1.6 - 2003/06/04 15:14:51

Generic File Queries

A generic file query is a powerful mechanism for locating files that match the criteria you specify. All the ready-made file queries that CScout provides you are just URLs specifying saved instances of generic queries.

You specify the query through the following form:

File Query

Writable
Read-only
Sort-byMetricCompareValue
Number of characters
Number of comment characters
Number of space characters
Number of line comments
Number of block comments
Number of lines
Maximum number of characters in a line
Number of character strings
Number of unprocessed lines
Number of C preprocessor directives
Number of processed C preprocessor conditionals (ifdef, if, elif)
Number of defined C preprocessor function-like macros
Number of defined C preprocessor object-like macros
Number of preprocessed tokens
Number of compiled tokens
Number of copies of the file
Number of statements
Number of defined project-scope functions
Number of defined file-scope (static) functions
Number of defined project-scope variables
Number of defined file-scope (static) variables
Number of complete aggregate (struct/union) declarations
Number of declared aggregate (struct/union) members
Number of complete enumeration declarations
Number of declared enumeration elements
Number of directly included files
Entity name

Reverse sort order

Match any of the above         Match all of the above


File names should match RE

Query title   

Main page - Web: Home Manual


CScout

You start by specifying whether the file should be writable (i.e. typically part of your application) and/or readable (i.e. typically part of the compiler or system). Next come a series of metrics CScout collects for each file. For each metric (e.g. the number of comments) you can specify an operator ==, !=, < or > and a number to match that metric against. Thus to locate files without any comments you would specify
Number of block comments == 0.

On the left of each metric you can specify whether that metric will be used to sort the resulting file list. In that case, the corresponding number will appear together with each file listed. A separate option allows you to specify that files should be sorted in the reverse order.

You can request to see files matching any of your specifications (Match any of the above) or to see files matching all your specifications (Match all of the above).

Sometimes you may only want to search in a subset of files; you can then specify a regular expression that filenames should match against (File names should match RE).

Finally, you can also specify a title for your query. The title will then appear on the result document annotating the results, and will also provide you with a sensible name when creating a bookmark to it.

C Namespaces

To understand identifier queries it is best to refresh our notion of the C namespaces. The main way we normally reuse identifier names in C programs is through scoping: an identifier within a given scope such as a block or declared as static within a file will not interfere with identifiers outside that scope. Thus, the following example will print 3 and not 7.
int i = 3;

foo()
{
        int i = 7;
}

main()
{
        foo();
        printf("%d\n", i);
}
CScout analyzes and stores each identifier's scope performing substitutions accordingly.

In addition, C also partitions a program's identifiers into four namespaces. Identifiers in one namespace, are also considered different from identifiers in another. The four namespaces are:

  1. Tags for a struct/union/enum
  2. Members of struct/union (actually a separate namespace is assigned to each struct/union)
  3. Labels
  4. Ordinary identifiers (termed objects in the C standard)
Thus in the following example all id identifier instances are different:
/* structure tag */
struct id {
        int id;                /* structure member */
};

/* Different structure */
struct id2 {
        char id;       /* structure member */
};

/* ordinary identifier */
id()
{
id:     /* label */
}
Furthermore, macro names and the names of macro formal arguments also live in separate namespaces within the preprocessor.

Normally when you want to locate or change an identifier name, you only consider identifiers in the same scope and namespace. Sometimes however, a C preprocessor macro can semantically unite identifiers living in different namespaces, so that changes in one of them should be propagated to the others. The most common case involves macros that access structure members.

struct s1 {
        int id;
} a;

struct s2 {
        char id;
} b;

#define getid(x) ((x)->id)

main()
{
        printf("%d %c", getid(a), getid(b));
}
In the above example, a name change in any of the id instances should be propagated to all others for the program to retain its original meaning. CScout understands such changes and will propagate any changes you specify accordingly.

Finally, the C preprocessor's token concatenation feature can result in identifiers that should be treated for substitution purposes in separate parts. Consider the following example:

int xleft, xright;
int ytop, ybottom;

#define coord(a, b) (a ## b)

main()
{
        printf("%d %d %d %d\n",
                coord(x, left),
                coord(x, right),
                coord(y, top),
                coord(y, bottom));
}
In the above example, replacing x in one of the coord macro invocations should replace the x part in the xleft and xright variables. Again CScout will recognize and correctly handle this code.

Identifier Elements

All identifier queries produce identifier lists data as their result. Clicking on an identifier in the list will lead you to a page like the following.

Identifier: copy_file

  • Read-only: No
  • Tag for struct/union/enum: No
  • Member of struct/union: No
  • Label: No
  • Ordinary identifier: Yes
  • Macro: No
  • Undefined macro: No
  • Macro argument: No
  • File scope: No
  • Project scope: Yes
  • Typedef: No
  • Enumeration constant: No
  • Function: Yes
  • Crosses file boundary: Yes
  • Unused: No
  • Matches 5 occurence(s)
  • Appears in project(s):
    • cp
  • The identifier occurs (wholy or in part) in function name(s):
    1. copy_file - function page

  • Substitute with:

Dependent Files (Writable)

DirectoryFile
/vol/src/bin/cp/ cp.cmarked source
/vol/src/bin/cp/ extern.hmarked source
/vol/src/bin/cp/ utils.cmarked source

Dependent Files (All)

DirectoryFile
/vol/src/bin/cp/ cp.cmarked source
/vol/src/bin/cp/ extern.hmarked source
/vol/src/bin/cp/ utils.cmarked source

Main page - Web: Home Manual


CScout 2.0 - 2004/07/31 12:37:12
As you see, for each identifier CScout will display: The substitution will globally replace the identifier (or the identifier part) in all namespaces, files, and scopes required for the program to retain its original meaning. No checks for name collisions are made, so ensure that the name you specify is unique for the appropriate scope. Performing the substitution operation will not change the identifier's name in the current invocation of CScout. However, once you have finished your browsing and replacing session, you have an option to terminate CScout and write back all the subtitutions you made to the respective source files.

Finally, the identifier's page will list the writable and all files the specific identifier appears in. Clicking on the ``marked source'' hyperlink will display the respective file's source code with only the given identifier marked as a hyperlink. By pressing your browser's tab key you can then see where the given identifier is used. In our example the cp.c source code with the copy_file identifier marked would appear as follows:
                case S_IFBLK:
                case S_IFCHR:
                        if (Rflag) {
                                if (copy_special(curr->fts_statp, !dne))
                                        badcp = rval = 1;
                        } else {
                                if (copy_file(curr, dne))
                                        badcp = rval = 1;
                        }
                        break;
                case S_IFIFO:
                        if (Rflag) {
                                if (copy_fifo(curr->fts_statp, !dne))
                                        badcp = rval = 1;
                        } else {
                                if (copy_file(curr, dne))
                                        badcp = rval = 1;
                        }
                        break;
                default:
                        if (copy_file(curr, dne))
                                badcp = rval = 1;
                        break;
                }

Identifier Metrics

The identifier metrics page displays a summary of metrics related to identifier use. In our example, the metrics are as follows:

Identifier Metrics

Writable Identifiers

Identifier classDistinct # idsTotal # idsAvg lengthMin lengthMax length
All identifiers1439104723.72272117
Tag for struct/union/enum10244.729
Member of struct/union567214.21429110
Label7165.2857129
Ordinary identifier114983423.51175117
Macro18512985.16216210
Undefined macro228.5611
Macro argument3069111
File scope15914935.47799217
Project scope27724426.28881212
Typedef97205.2222228
Enumeration constant00-00
Function17714556.58192317

Read-only Identifiers

Identifier classDistinct # idsTotal # idsAvg lengthMin lengthMax length
All identifiers37511806.36118
Tag for struct/union/enum5145.427
Member of struct/union56677.80357217
Label00-00
Ordinary identifier1646046.04878314
Macro1164097.47414218
Undefined macro171549415
Macro argument33831.4242414
File scope121317.25414
Project scope1524735.95395313
Typedef121317.25414
Enumeration constant00-00
Function1464495.84247310

Main page - Web: Home Manual


CScout

You can use these metrics to compare characteristics of different projects, adherance to coding standards, or to identify identifier classes with abnormally short or long names. The ratio between the distinct number of identifiers and the total number of identifiers is the number of times each identifier is used. Notice the difference in our case between the read-only identifiers (which are mostly declarations) and the writable identifiers (which are actually used).

All identifiers

The all identifiers page will list all the identifiers in your project in alphabetical sequence. In large projects this page will be huge.

Read-only identifiers

The ``read-only identifiers'' page will only list the read-only identifiers of your project in alphabetical sequence. These typically become part of the project through included header files.

Writable identifiers

The ``writable identifiers'' page will only list the writable identifiers of your project in alphabetical sequence. These are typically the identifiers your project has defined. In large projects this page will be huge.

File-spanning writable identifiers

The ``file-spanning writable identifiers'' page will only list your project's identifiers that span a file boundary. Refactoring operations and coding standards typically pay higher attention to such identifiers, since they tend occupy the project's global namespace. In our example, the following page is generated:

File-spanning Writable Identifiers

Matching Identifiers

PATH_T
arg
copy_fifo
copy_file
copy_link
copy_special
fflag
iflag
netsettime
nflag
p_end
p_path
pflag
setfile
target_end
to
usage
vary
vary_append
vary_apply
vary_destroy
vflag

You can bookmark this page to save the respective query

Main page


CScout 1.6 - 2003/06/04 15:14:51

Unused project-scoped writable identifiers

The unused project-scoped writable identifiers are useful to know, since they can pinpoint functions or variables that can be eliminated from a workspace.

Unused file-scoped writable identifiers

The unused file-scoped writable identifiers can also pinpoint functions or variables that can be eliminated from a file. In our example the following list is generated:

Unused File-scoped Writable Identifiers

Matching Identifiers

copyright
copyright
copyright
rcsid
rcsid
rcsid
rcsid

You can bookmark this page to save the respective query

Main page


CScout 1.6 - 2003/06/04 15:14:51
Notice how distinct identifiers appear as separate entries.

Unused writable macros

Finally, the unused writable macros page will list macros that are not used within a workspace. In our case the list contains an identifier that was probably used in an earlier version.

Unused Writable Macros

Matching Identifiers

RETAINBITS

You can bookmark this page to save the respective query

Main page


CScout 1.6 - 2003/06/04 15:14:51

Generic Identifier Queries

The generic identifier query feature of CScout is one of its most powerfull features, allowing you to accurately specify the properties of identifiers you are looking for, by means of the following form.

Identifier Query

Writable
Read-only
Tag for struct/union/enum
Member of struct/union
Label
Ordinary identifier
Macro
Undefined macro
Macro argument
File scope
Project scope
Typedef
Enumeration constant
Crosses file boundary
Unused

Match any marked         Match all marked         Exclude marked         Exact match


Identifier names should ( not) match RE
Select identifiers from filenames matching RE

Query title   

Main page


CScout 1.16 - 2003/08/17 12:13:01
In the form you specify: You can either select to see the identifiers that match the specific query, or the files containing identifiers that match the query. In the second case, each file in the file list will provide you with a link (marked source) showing the file's source code with all matched identifiers marked using hyperlinks.

As an example, the following query could be used to identify unused file-scoped writable identifiers, but excluding the copyright and rcsid identifiers:

Identifier Query

Writable
Read-only
Tag for struct/union/enum
Member of struct/union
Label
Ordinary identifier
Macro
Undefined macro
Macro argument
File scope
Project scope
Typedef
Enumeration constant
Crosses file boundary
Unused

Match any marked         Match all marked         Exclude marked         Exact match


Identifier names should ( not) match RE
Select identifiers from filenames matching RE

Query title   

Main page


CScout 1.16 - 2003/08/17 12:13:01

Function Elements

Every function (C function or function like macro) is associated with a page like the following.

Function: format (C function)

Details

Metrics

MetricValue
Number of characters3237
Number of comment characters204
Number of space characters767
Number of line comments0
Number of block comments7
Number of lines133
Maximum number of characters in a line95
Number of character strings15
Number of unprocessed lines0
Number of C preprocessor directives0
Number of processed C preprocessor conditionals (ifdef, if, elif)0
Number of defined C preprocessor function-like macros0
Number of defined C preprocessor object-like macros0
Number of preprocessed tokens962
Number of compiled tokens1012
Number of statements or declarations113
Number of operators176
Number of unique operators15
Number of numeric constants22
Number of character literals43
Number of if statements17
Number of else clauses2
Number of switch statements2
Number of case labels19
Number of default labels2
Number of break statements14
Number of for statements2
Number of while statements1
Number of do statements0
Number of continue statements2
Number of goto statements0
Number of return statements1
Number of project-scope identifiers53
Number of file-scope (static) identifiers2
Number of macro identifiers9
Total number of object and object-like identifiers259
Number of unique project-scope identifiers12
Number of unique file-scope (static) identifiers2
Number of unique macro identifiers5
Number of unique object and object-like identifiers34
Number of global namespace occupants at function's top1063
Number of parameters4
Maximum level of statement nesting4
Number of goto labels0
Fan-in (number of calling functions)2
Fan-out (number of called functions)14
Cyclomatic complexity (control statements)23
Extended cyclomatic complexity (includes branching operators)27
Maximum cyclomatic complexity (includes branching operators and all switch branches)44
Structure complexity (Henry and Kafura)784
Halstead complexity3416.45
Information flow metric (Henry and Selig)18032

Main page — Web: Home Manual


CScout

From this page you can refactor the function's arguments (more on this in the next section) and obtain the following data.

All Functions

The all functions page will list all the functions (C functions and function-like macros) defined or declared in the CScout workspace. In moderately sized projects, you can use it as a starting point for jumping to a function; in larger projects it is probably useful only as a last resort.

Project-scoped writable functions

This page contains all the writable functions that are globaly visible. The page does not list function-like macros.

File-scoped writable functions

This page contains all the writable functions that are visible only within the context of a single file. This include C functions declared as static, and function-like macros.

Writable functions that are not directly called

This page will list all writable functions that are never directly called. The most probable cause is that the corresponding functions are called through a pointer, but some may be historic leftovers - candidates for removal.

Writable functions that are called exactly once

Functions that are called exactly once may be candidates for inlining.

Refactoring Function Arguments

A text box appearing on a function's page allows the refactoring of a function's arguments across all identified calls of the function. This box will appear only for functions whose identifiers are writable (i.e. all instances of them appear in writable files), and where there is a one to one correspondence between the function name and the corresponding identifier. If the same identifier is aliased through a macro to refer to various functions or if a function's name is generated by pasting together multiple identifiers, then the function argument refactoring facility will not be made available for that function. The requirement for the function's identifier to be writable can be overridden through the options page.

To refactor the function's arguments, one simply enters in the text box a template describing the argument replacement pattern. The template consists of text, which is copied verbatim as a function's argument, and elements starting with the operator @, which have a special meaning. The combined effect of this template mechanism allows you to

The following types of @ operator are supported. N is always an integer starting from 1, and denotes the function's Nth argument.
@N
pastes the original Nth argument passed to the function. Thus, @1 will get replaced with the function's first argument. Specifying in a template for a function taking two arguments "@2, @1" will swap their order, while specifying "@1, sizeof(@1), stdin" as the arguments for gets will refactor them in a form suitable for calling fgets (if the original argument refers to a fixed-size character array).
@.N
pastes the Nth argument and all subsequent ones, separated by commas. This is useful for handling functions with a variable number of arguments, like printf. Specifying in a template for the printf function "stdout, @1, @.2" will introduce an extra first parameter, named stdout. (Presumably the function will also be renamed to fprintf.)
@+N{...}
pastes the text in the braces, if the specific function being replaced has an Nth argument. The text in the braces can include arbitrary text, including nested @ operators.
@-N{...}
pastes the text in the braces, if the specific function being replaced does not have an Nth argument.
The last two operators can often be combined to achieve more complex results. For instance, the template "@1, @2, @+3{@3}@-3{NULL}" will add to any call to the function missing a third argument, a third argument with a value of NULL.

Note that the refactorings will take place on all instances where the identifier is found to match the function or macro. This includes declarations and definitions (which might require some hand-editing if arguments are introduced), and the appearance of the name in the replacement text of a macro, when that macro is used in a way that makes the function match the one being refactored. The replacements will not be performed to function calls that are executed through a function pointer.

Call Graphs

CScout can create call graphs that list how functions call each other. Keep in mind that the graphs only indicate the calls detected by statically analyzing the program source. Calls via function pointers will not appear in the call graph.

Two global options specify the format of the call graph and the content on each graph's node. Through these options you can obtain graphs in

All diagrams follow the notation
calling function -> called function

Two links on the main page (function and macro call graph, and non-static function call graph) can give you the call graphs of the complete program. For any program larger than a few thousand lines, these graphs are only useful in their textual form. In their graphical form, even with node information disabled, they can only serve to give you a rough idea of how the program is structured. The following image depicts how the three different programs we analyzed in the bin example relate to each other.

More useful are the call graphs that can be generated for individual functions. These can allow you to see what paths can possibly lead to a given function (call graph of all callers) and which functions can be reached starting from a given function.

As an example, the following diagram depicts all paths leading to the setfile function.

Correspondingly, the functions that can be reached starting from the copy_file function appears in the following diagram.

Generic Function Queries

The generic function query allows you to select functions by means of the following form.

Function Query

C function
Function-like macro
Writable declaration
Read-only declaration
Project scope
File scope
Defined
Sort-byMetricCompareValue
Number of characters
Number of comment characters
Number of space characters
Number of line comments
Number of block comments
Number of lines
Maximum number of characters in a line
Number of character strings
Number of unprocessed lines
Number of C preprocessor directives
Number of processed C preprocessor conditionals (ifdef, if, elif)
Number of defined C preprocessor function-like macros
Number of defined C preprocessor object-like macros
Number of preprocessed tokens
Number of compiled tokens
Number of statements or declarations
Number of operators
Number of unique operators
Number of numeric constants
Number of character literals
Number of if statements
Number of else clauses
Number of switch statements
Number of case labels
Number of default labels
Number of break statements
Number of for statements
Number of while statements
Number of do statements
Number of continue statements
Number of goto statements
Number of return statements
Number of project-scope identifiers
Number of file-scope (static) identifiers
Number of macro identifiers
Total number of object and object-like identifiers
Number of unique project-scope identifiers
Number of unique file-scope (static) identifiers
Number of unique macro identifiers
Number of unique object and object-like identifiers
Number of global namespace occupants at function's top
Number of parameters
Maximum level of statement nesting
Number of goto labels
Fan-in (number of calling functions)
Fan-out (number of called functions)
Cyclomatic complexity (control statements)
Extended cyclomatic complexity (includes branching operators)
Maximum cyclomatic complexity (includes branching operators and all switch branches)
Structure complexity (Henry and Kafura)
Halstead complexity
Information flow metric (Henry and Selig)
Entity name

Reverse sort order

Match any marked         Match all marked         Exclude marked         Exact match


Number of direct callers
Function names should ( not) match RE
Names of calling functions should ( not) match RE
Names of called functions should ( not) match RE
Select functions from filenames matching RE

Query title   

Main page - Web: Home Manual


CScout

On the top you can specify whether each function you want listed:

As is the case in file queries, next comes a series of metrics CScout collects for each defined function. For each metric (e.g. the number of comments) you can specify an operator ==, !=, < or > and a number to match that metric against. Thus to locate functions containing goto statement you would specify
Number of goto statements != 0.

On the left of each metric you can specify whether that metric will be used to sort the resulting file list. In that case, the corresponding number will appear together with each file listed. A separate option allows you to specify that files should be sorted in the reverse order.

Similarly to the identifier query, you can also specify whether the specified properties should be treated

In addition you can specify:

Global Options

The operations CScout provides group together functions that globally affect its operation. The global options link leads you to the following page.

Global Options

File and Identifier Pages
Show only true identifier classes (brief view)
Show associated projects
Show a list of identical files
Source Listings
Show line numbers
Tab width
Refactoring
Allow the renaming of read-only identifiers
Allow the refactoring of function arguments of read-only functions
Queries
Case-insensitive file name regular expression match
Query Result Lists
Number of entries on a page
Show file lists with file name in context
Sort identifiers starting from their last character
Call Graphs
Call graph links should lead to pages of:plain text HTML dot SVG (via dot) GIF (via dot)
Call graphs should contain:only edges function names file and function names path and function names
Maximum number of call levels in a graph
Saved Files
When saving modified files replace RE
... with the string
Editing
External editor invocation command

Main page — Web: Home Manual


CScout
The meaning of each option is described in the following sections.

File and Identifier Pages

Show Only True Identifier Classes

Setting the option ``show only true identifier classes (brief view)'' will remove from each identifier page all identifier properties marked as no, resulting in a less verbose page.

Identifier: argc

  • Ordinary identifier: Yes
  • Matches 8 occurence(s)
  • Appears in project(s):
    • cp
  • Substitute with:

Dependent Files (Writable)

DirectoryFile
/vol/src/bin/cp/ cp.cmarked source

Dependent Files (All)

DirectoryFile
/vol/src/bin/cp/ cp.cmarked source

Main page - Web: Home Manual


CScout 2.0 - 2004/07/31 12:37:12

Show Associated Projects

Normally, each identifier or file page lists the projects in which the corresponding identifier or file has appeared during processing. When the CScout workspace typically consists only of a single project, or consists of hundreds of projects, listing the project membership can be useless or result into too volumneous output. The corresponding option can be used to control this output.

Show Lists of Identical Files

CScout will detect during processing when a file is an exact duplicate of another file (typically the result of a copy operation during the building process). On the file information page it will then list the files that are duplicates of the one being listed. The corresponding option can be used to control this output.

Source Listings

Show Line Numbers

The "show line numbers in source listings" option allows you to specify whether the source file line numbers will be shown in source listings. Line numbers can be useful when you are editing or viewing the same file with an editor. A file with line numbers shown appears as follows:

   78 fa *makedfa(const char *s, int anchor)  /* returns dfa for reg expr s */
   79 {
   80         int i, use, nuse;
   81         fa *pfa;
   82         static int now = 1;
   83 
   84         if (setvec == 0) {      /* first time through any RE */
   85                 maxsetvec = MAXLIN;
   86                 setvec = (int *) malloc(maxsetvec * sizeof(int));
   87                 tmpset = (int *) malloc(maxsetvec * sizeof(int));
   88                 if (setvec == 0 || tmpset == 0)
   89                         overflo("out of space initializing makedfa");
   90         }
   91 
   92         if (compile_time)       /* a constant for sure */
   93                 return mkdfa(s, anchor);
   94         for (i = 0; i < nfatab; i++)    /* is it there already? */
   95                 if (fatab[i]->anchor == anchor
   96                   && strcmp((const char *) fatab[i]->restr, s) == 0) {
   97                         fatab[i]->use = now++;
   98                         return fatab[i];
   99                 }
  100         pfa = mkdfa(s, anchor);
  101         if (nfatab < NFA) {     /* room for another */
  102                 fatab[nfatab] = pfa;
  103                 fatab[nfatab]->use = now++;
  104                 nfatab++;
  105                 return pfa;
  106         }

Tab Width

The ``code listing tab width'' option allows you to specify the tab width to use when listing source files as hypertext (8 by default). The width should match the width normally used to display the file. It does not affect the way the modified file is written; tabs and spaces will get written exactly as found in the source code file.

Refactoring

Allow the renaming of read-only identifiers

Setting this option will present a rename identifier box, in an identifier's page, even if that identifier occurs in read-only files. When CScout exist saving refactoring changes, replacements in those files may fail due to file system permissions.

Allow the refactoring of function arguments of read-only functions

Setting this option will present a function argument refactoring template input box in an function's page, even if that identifier associated with the function occurs in read-only files.

Queries

Case-insensitive File Name Regular Expression Match

Some environments, such as Microsoft Windows, are matching filenames in a case insensitive manner. As a result the same filename may appear with different capitalization (e.g. Windows.h, WINDOWS.h, and windows.h). The use of the ``case-insensitive file name regular expression match'' option makes filename regular expression matches ignore letter case thereby matching the operating system's semantics.

Query Result Lists

Number of Entries on a Page

The number of entries on a page, specifies the number of records appearing on each separate page resulting from a file, identifier, or function query. Too large values of this option (say above 1000) may cause your web browser to behave sluggishly, and will also reduce the program's responsiveness when operating over low-bandwidth network links.

Show File Lists With File Name in Context

Setting the ``Show file lists with file name in context'' option will result in file lists showing the file name (the last component of the complete path) in the same position, as in the following example:

Read-only Files

DirectoryFile
/usr/include/ ctype.h
/usr/include/ err.h
/usr/include/ errno.h
/usr/include/ fcntl.h
/usr/include/ fts.h
/usr/include/ limits.h
/usr/include/ locale.h
/usr/include/machine/ ansi.h
/usr/include/machine/ endian.h
/usr/include/machine/ limits.h
/usr/include/machine/ param.h
/usr/include/machine/ signal.h
/usr/include/machine/ trap.h
/usr/include/machine/ types.h
/usr/include/machine/ ucontext.h
/usr/include/ runetype.h
/usr/include/ stdio.h
/usr/include/ stdlib.h
/usr/include/ string.h
/usr/include/sys/ _posix.h
/usr/include/sys/ cdefs.h
/usr/include/sys/ inttypes.h
/usr/include/sys/ param.h
/usr/include/sys/ signal.h
/usr/include/sys/ stat.h
/usr/include/sys/ syslimits.h
/usr/include/sys/ time.h
/usr/include/sys/ types.h
/usr/include/sys/ ucontext.h
/usr/include/sys/ unistd.h
/usr/include/ sysexits.h
/usr/include/ syslog.h
/usr/include/ time.h
/usr/include/ unistd.h

You can bookmark this page to save the respective query

Main page

This results in lists that are easier to read, but that can not be easilly copy-pasted into other tools for further processing.

Sort Identifiers Starting from their Last character

Some coding conventions use identifier suffixes for distinguishing the use of a given identifier. As an example, typedef identifiers often end in _t. The following list contains our example's typedefs ordered by the last character, making it easy to distinguish typedefs not ending in _t
FILE
FTS
FTSENT
PATH_T
_RuneRange
_RuneLocale
u_long
fd_mask
u_char
physadr
int32_t
__int32_t
u_int32_t
uint32_t
__uint32_t
inthand2_t
ointhand2_t
int64_t
[... 40 lines removed]
in_addr_t
caddr_t
c_caddr_t
v_caddr_t
daddr_t
ufs_daddr_t
u_daddr_t
qaddr_t
__sighandler_t
__siginfohandler_t
timer_t
register_t
u_register_t
intptr_t
__intptr_t
uintptr_t
__uintptr_t
fpos_t
timecounter_pps_t
timecounter_get_t
vm_offset_t
vm_ooffset_t
sigset_t
osigset_t
fixpt_t
in_port_t
mcontext_t
ucontext_t
dev_t
div_t
ldiv_t
vm_pindex_t
key_t
segsz_t
fd_set
u_int
uint
u_short
ushort
_RuneEntry

Call Graphs

Call Graph Links Should Lead to Pages of

Function and macro call graphs can appear in four different formats.

Call Graphs Should Contain

This option allows you to specify the level of detail you wish to see in the call graph nodes.

Maximum number of call levels in a graph

Call graphs can easily grow too large for viewing, printing, or even formatting as a graph. This option limits the number of functions that will be traversed from a caller when computing a call graph or a list of calling or called functions.

Saved Files

When Saving Modified Files Replace

When saving files where an identifier has been modified it is often useful to use a different directory than the one where the original version of the source code resides. This allows you to To use this option, specify a regular expression that will match a path component of the original source code files (often just a fixed string), and a corresponding substitution string. As an example, if your project files are of the type /home/jack/src/foo/filename.c, you could specify that /foo/ should be changed into /../foo.new/.

Note than when this option is specified the existing and new locations of the file must reside on the same drive and partition (under Windows) or file system (under Unix).

Editing

The "External editor invocation command" allows the specification of the editor that wil be used for hand-editing files. This string can contain two %s placeholders. The first is substituted by a regular expression that is associated with the identifier for which the file is edited, while the second is substituted with the corresponding file name. The default string under Unix is and under Windows Under Windows a more sensible default could be something like which fires off the VIM editor in a new window.

Option Files

The link on the right of global options allows you to save the CScout global options into a file. A directory .cscout will be created in the current directory (if it does not already exist), and a file named options will be written in it, listing the options you specified.

When CScout starts-up it will attempt to load the options file by searching in $CSCOUT_HOME, $HOME/.cscout, or .cscout in the current directory.

The options file is text based and contains key-value pairs. The order of the entries is not significant. This is an example of an options file.

fname_in_context: 1
sort_rev: 0
show_true: 1
show_line_number: 0
file_icase: 0
cgraph_type: s
cgraph_show: n
tab_width: 8

Operations

The operations CScout provides group together functions that globally affect its operation. The following sections describe all operations appart from the global options.

Identifier Replacements

This operation allows you to review the identifier replacements you have specified in identifier pages, and modify or selectively deactivate some of them. This page, together with the "save and continue" operation and the file path substitution option provide you a way to test and revoke source code changes, while operating CScout.

The following is an example of the identifier replacements page. You see all identifiers for which replacements have been specified. All specified replacements are originally active. If a particular replacement appears to be causing problems you can deactivate it from this page. In addition, you can change the replaced name of any of the replaced identifiers. Finally, clicking on an identifier name will lead you to the corresponding identifier page.

Identifier Replacements

IdentifierReplacementActive
setfile
netsettime

Main page - Web: Home Manual


CScout 2.0 - 2004/07/31 12:37:12

Select Active Project

When using a workspace with multiple projects, you can restrict the results of all identifier and file queries (read-made and those you explicitly specify) to refer to a particular project or to all projects. The metric results displayed are not affected. When a project is delected, all pages end with a remark indicating the fact. The following shows our example's project selection page.

Select Active Project

Project cp is currently selected

Main page


CScout 1.6 - 2003/06/04 15:14:51

Save Changes and Continue

Through this option you can save changes you have made to the program's identifiers, and continue CScout's operation. CScout bases its source code display facilities on the source code it has analyzed. Therefore, this operation can only be executed if a file substitution regular expression has been specified as an option.

Exit - Saving Changes

Once you have changed the name of some identifiers by substituting it with another name, you should exit CScout through this option to commit the changes you made to the respective file source code.

Exit - Ignore Changes

You can also exit CScout without committing any changes. As this option will trigger millions of object desctructors in large workspaces, it may be faster to just terminate CScout from its command-line instance by pressing ^C.

Hand-Editing

Some file and identifier listings provide an option to edit the file by hand. Such an operation is useful when CScout has identified a function as unused, and one therefore wishes to remove the complete function body. The edit link invokes an external editor, where possible with an argument that will move the edit point near the point of the corresponding identifier. The argument is specified as a regular expression. This has the advantage that the location will work even when the file length changes, but the disadvantage is imprecise and can also result in spurious matches.

The automatic global identifier replacement and the hand-editing of files are mutualy exclusive operations. Once either of the two is performed the other ceases to be available. This is done to protect the integrity of the underlying source code. Furthermore, all CScout's operations, such as queries and source code listings, are always performed on a snapshot of the source code taken just before a file is edited by hand.

Interfacing with Version Management Systems

When the files CScout will modify are under revision control you may want to check them out for editing before doing the identifier substitutions, and then check them in again. CScout provides hooks for this operation. Before a file is modified CScout will try to execute the command cscout_checkout; after the file is modified CScout will try to execute the command cscout_checkin. Both commands will receive as their argument the full path name of the respective file. If commands with such names are in your path, they will be executed performing whatever action you require.

As an example, for a file under RCS control the following commands could be used:

cscout_checkout

#!/bin/sh
co -l $1

cscout_checkin

#!/bin/sh
co -m 'CScout identifier name refactoring' -u $1

Language Extensions

CScout implements the C language as defined in ANSI X3.159-1989. In addition, it supports the following extensions:
  1. Preprocessor directives can appear within a call to a function-like macro (gcc)
  2. Initializers and compound literals can be empty (gcc)
  3. The alignof operator can be used on types (gcc)
  4. A declaration expression as the first element of a for statement (C99)
  5. The restrict qualifier and the inline specifier (C99)
  6. Initialization designators (C99)
  7. Array initialization designators can include ranges (gcc)
  8. ANSI-style function definitions can be nested (gcc) (gcc also allows nested K&R-style function definitions)
  9. The equals sign following an initializer designator is optional (gcc)
  10. Array and structure initialization (gcc)
  11. Compound literals (C99)
  12. Declarations can be intermixed with statements (C99).
  13. Recognise __atribute__(__unused__) for determining which identifiers should not be reported as unused (gcc).
  14. // line comments (common extension)
  15. __asm__ blocks (gcc)
  16. enum lists ending with a comma (common extension)
  17. Anonymous struct/union members (gcc, Microsoft C)
  18. Allow case expression ranges (gcc).
  19. An enumeration list can be empty (Microsoft C)
  20. Allow braces around scalar initializers (common extension).
  21. Indirect goto targets and the label address-of operator (gcc).
  22. __typeof keyword (gcc)
  23. A compound statement in brackets can be an expression (gcc)
  24. Macros expanding from /##/ into // are then treated as a line comment (Microsoft C)
  25. Exception handling using the __try __except __finally __leave keywords (Microsoft C)
  26. #include_next preprocessor directive (gcc)
  27. #warning preprocessor directive (gcc)
  28. Variable number of arguments preprocessor macros (support for both the gcc and the C99 syntax)
  29. Allow empty member declarations in aggregates (gcc)
  30. long long type (common extension)
  31. A semicolon can appear as a declatation (common extension)
  32. An aggregate declaration body can be empty (gcc)
  33. Aggregate member initialization using the member: value syntax (gcc)
  34. Statement labels do not require a statement following them (gcc)
  35. #ident preprocessor directive (gcc)
  36. Allow assignment to case expressions (common extension)
  37. Accept an empty translation unit (common extension).
  38. Support locally declared labels (__label__) (gcc).
  39. The second argument of a conditional expression can be omitted (gcc).
  40. Dereferencing a function yields a function (common extension).
Many other compiler-specific extensions are handled by suitable macro definitions in the CScout initialization file.

Processing Yacc Files

Many C programs include parsing code in the form of yacc source files. CScout can directly process those files, allowing you to analyze and modify the identifiers used in those files. CScout determines whether a file is yacc source or plain C, by examining the file's suffix: file names ending in a lowercase 'y' are considered to contain yacc source and processed accordingly.

CScout processes yacc files as follows:

CScout is designed to process well-formed modern-style yacc files. All rules must be terminated with a semicolon (apparently this is optional in the original yacc version). The accepted grammar appears below.

body:
	defs '%%' rules tail
	;

tail:
	/* Empty */
	| '%%' c_code
	;

defs:
	/* Empty */
	| defs def
	;

def:
	'%start' IDENTIFIER
	| '%union' '{' member_declaration_list  '}' 
	| '%{' c_code '%}'
	| rword name_list_declaration
	;

rword:
	'%token'
	| '%left'
	| '%right'
	| '%nonassoc'
	| '%type'
	;

tag:
	/* Empty: union tag is optional */
	| '<' IDENTIFIER '>'
	;

name_list_declaration:
	tag name_number
	| name_list_declaration opt_comma name_number
	;

opt_comma:
	/* Empty */
	| ','
	;

name_number:
	name
	| name INT_CONST
	;

name:
	IDENTIFIER
	| CHAR_LITERAL
	;

/* rules section */

rules:
	rule
	| rules rule
	;

rule:
	IDENTIFIER ':'  rule_body_list ';'
	;

rule_body_list:
	rule_body
	| rule_body_list '|' rule_body
	;

rule_body:
	id_action_list prec
	;

id_action_list:
	/* Empty */
	| id_action_list name
        | id_action_list '{' c_code '}' 
	;

prec:
	/* Empty */
	| '%prec' name
	| '%prec' name  '{' c_code '}'
	;

variable:
	'$$'
	| '$' INT_CONST
	| '$-' INT_CONST
		{ $$ = basic(b_int); }
	| '$<' IDENTIFIER '>' variable_suffix
		{ $$ = $3; }
	;

variable_suffix:
	'$'
	| INT_CONST
	| '-' INT_CONST
	;

Regular Expression Syntax

CScout allows you to specify regular expressions for specifying identifier or file names you are looking for. The following description of the regular expressions CScout accepts is adapted from the FreeBSD re_format(7) manual page.

Regular expressions (``REs''), as defined in IEEE Std 1003.2 (``POSIX.2''), come in two forms: modern REs (roughly those of egrep(1); 1003.2 calls these ``extended'' REs) and obsolete REs (roughly those of ed(1); 1003.2 ``basic'' REs). CScout has adopted the use of modern (extended) REs.

A (modern) RE is one= or more non-empty= branches, separated by `|'. It matches anything that matches one of the branches.

A branch is one= or more pieces, concatenated. It matches a match for the first, followed by a match for the second, etc.

A piece is an atom possibly followed by a single= `*', `+', `?', or bound. An atom followed by `*' matches a sequence of 0 or more matches of the atom. An atom followed by `+' matches a sequence of 1 or more matches of the atom. An atom followed by `?' matches a sequence of 0 or 1 matches of the atom.

A bound is `{' followed by an unsigned decimal integer, possibly followed by `,' possibly followed by another unsigned decimal integer, always fol- lowed by `}'. The integers must lie between 0 and RE_DUP_MAX (255=) inclusive, and if there are two of them, the first may not exceed the second. An atom followed by a bound containing one integer i and no comma matches a sequence of exactly i matches of the atom. An atom fol- lowed by a bound containing one integer i and a comma matches a sequence of i or more matches of the atom. An atom followed by a bound containing two integers i and j matches a sequence of i through j (inclusive) matches of the atom.

An atom is a regular expression enclosed in `()' (matching a match for the regular expression), an empty set of `()' (matching the null string)=, a bracket expression (see below), `.' (matching any single character), `^' (matching the null string at the beginning of a line), `$' (matching the null string at the end of a line), a `\' followed by one of the characters `^.[$()|*+?{\' (matching that character taken as an ordinary character), a `\' followed by any other character= (matching that character taken as an ordinary character, as if the `\' had not been present=), or a single character with no other significance (matching that character). A `{' followed by a character other than a digit is an ordinary character, not the beginning of a bound=. It is illegal to end an RE with `\'.

A bracket expression is a list of characters enclosed in `[]'. It nor- mally matches any single character from the list (but see below). If the list begins with `^', it matches any single character (but see below) not from the rest of the list. If two characters in the list are separated by `-', this is shorthand for the full range of characters between those two (inclusive) in the collating sequence, e.g. `[0-9]' in ASCII matches any decimal digit. It is illegal= for two ranges to share an endpoint, e.g. `a-c-e'. Ranges are very collating-sequence-dependent, and portable programs should avoid relying on them.

To include a literal `]' in the list, make it the first character (fol- lowing a possible `^'). To include a literal `-', make it the first or last character, or the second endpoint of a range. To use a literal `-' as the first endpoint of a range, enclose it in `[.' and `.]' to make it a collating element (see below). With the exception of these and some combinations using `[' (see next paragraphs), all other special charac- ters, including `\', lose their special significance within a bracket expression.

Within a bracket expression, a collating element (a character, a multi- character sequence that collates as if it were a single character, or a collating-sequence name for either) enclosed in `[.' and `.]' stands for the sequence of characters of that collating element. The sequence is a single element of the bracket expression's list. A bracket expression containing a multi-character collating element can thus match more than one character, e.g. if the collating sequence includes a `ch' collating element, then the RE `[[.ch.]]*c' matches the first five characters of `chchcc'.

Within a bracket expression, a collating element enclosed in `[=' and `=]' is an equivalence class, standing for the sequences of characters of all collating elements equivalent to that one, including itself. (If there are no other equivalent collating elements, the treatment is as if the enclosing delimiters were `[.' and `.]'.) For example, if `x' and `y' are the members of an equivalence class, then `[[=x=]]', `[[=y=]]', and `[xy]' are all synonymous. An equivalence class may not= be an end- point of a range.

Within a bracket expression, the name of a character class enclosed in `[:' and `:]' stands for the list of all characters belonging to that class. Standard character class names are:

   alnum    digit    punct
   alpha    graph    space
   blank    lower    upper
   cntrl    print    xdigit
These stand for the character classes defined in ctype(3). A locale may provide others. A character class may not be used as an endpoint of a range.

There are two special cases= of bracket expressions: the bracket expres- sions `[[:<:]]' and `[[:>:]]' match the null string at the beginning and end of a word respectively. A word is defined as a sequence of word characters which is neither preceded nor followed by word characters. A word character is an alnum character (as defined by ctype(3)) or an underscore. This is an extension, compatible with but not specified by IEEE Std 1003.2 (``POSIX.2''), and should be used with caution in soft- ware intended to be portable to other systems.

In the event that an RE could match more than one substring of a given string, the RE matches the one starting earliest in the string. If the RE could match more than one substring starting at that point, it matches the longest. Subexpressions also match the longest possible substrings, subject to the constraint that the whole match be as long as possible, with subexpressions starting earlier in the RE taking priority over ones starting later. Note that higher-level subexpressions thus take priority over their lower-level component subexpressions.

Match lengths are measured in characters, not collating elements. A null string is considered longer than no match at all. For example, `bb*' matches the three middle characters of `abbbc', `(wee|week)(knights|nights)' matches all ten characters of `weeknights', when `(.*).*' is matched against `abc' the parenthesized subexpression matches all three characters, and when `(a*)*' is matched against `bc' both the whole RE and the parenthesized subexpression match the null string.

If case-independent matching is specified, the effect is much as if all case distinctions had vanished from the alphabet. When an alphabetic that exists in multiple cases appears as an ordinary character outside a bracket expression, it is effectively transformed into a bracket expres- sion containing both cases, e.g. `x' becomes `[xX]'. When it appears inside a bracket expression, all case counterparts of it are added to the bracket expression, so that (e.g.) `[x]' becomes `[xX]' and `[^x]' becomes `[^xX]'.

Access Control

The unsupported version of CScout allows any machine from the Internet to connect to your server for casual browsing. Operations requiring substantial CPU resources, or operations that will modify source files or will change the CScout execution environment can only be performed from the local host.

The supported version of CScout features an access control list. The list is specified in a file called acl which should be located in $CSCOUT_HOME, $HOME/.cscout, or .cscout in the current directory. The list contains lines with IP numeric addresses prefixed by an A (allow) or D (deny) prefix and a space. Matching is performed by comparing a substring of a machine's IP address against the specified access rule. Thus an entry such as

A 128.135.11.
can be used to allow access from a whole subnet. Unfortunatelly allowing access from the IP address 192.168.1.1 will also allow access 192.168.1.10, 192.168.1.100, and so on. Allow and deny entries cannot be combined in a useful manner since the rules followed are: Thus you will either specify a restricted list of allowed hosts, or allow access to the world, specifying a list of denied hosts.

Obfuscation Back-end

The supported version of CScout can convert a workspace into an obfuscated version. The obfuscated version of the workspace can be distributed instead of the original C source, and can be compiled on different processor architectures and operating systems, hindering however the code's reverse engineering and modification.

Each source code file is obfuscated by

Before running CScout to obfuscate, make a complete backup copy of your source code files, and store them in a secure place; preferably off-line. Once the source code files are obfuscated and overwritten, there is no way to get back their original contents.

To obfuscate the workspace, first ensure that CScout can correctly process the complete set of its source code files. Use the "unprocessed lines" metric of each file to verify that no parts of a file are left unprocessed; unprocessed regions will not be obfuscated. You can easily increase the coverage of CScout's processing by including in the workspace multiple projects with different defined directives.

Also ensure that all your project's files are considered writable, and no files outside your project (for example system headers) are considered writable. This will allow CScout to rename your identifier names, but keep the names of library-defined identifiers (for example printf) unchanged.

Finally, run CScout with the switch -o. For each writable workspace file CScout will create a file ending in .obf that will contain the obfuscated version of its contents. The files are not overwritten, providing you with another countermeasure against accidentally destroying them. To overwrite the original source with the obfuscated one, use the following Unix command:

find . -name '*.obf' |
sed 's/\\/\//g;s/\(.*\)\.obf$/mv "\1.obf" "\1"/' |
sh
You can then compile the obfuscated version of your project, to verify the obfuscation's results.

SQL Back-end

The supported version of CScout can dump the relationships of an entire workspace in the form of a SQL script. This can then be uploaded into a relational database for further querying and processing.

To generate the SQL script simply run CScout with the switch -s dialect, where the argument specifies the SQL dialect (for example, mysql, or postgresql). The SQL script will appear in CScout's standard output, allowing you to directly pipe the results into the database's client. For example, say the database you would want to create for your project was called myproj.
For MySQL you would write:

(
        echo "create database myproj; use myproj ;"
        cscout -s mysql myproj.cs
) | mysql
For PostgreSQL you would write:
createdb -U username myproj
cscout -s postgres myproj.cs | psql -U username myproj
For HSQLDB you would write:
cscout -s hsqldb myproj.cs |
java -classpath $HSQLDBHOME/lib/hsqldb/hsqldb.jar org.hsqldb.util.SqlTool --rcfile $HSQLDBHOME/lib/hsqldb/sqltool.rc mem -
The direct piping allows you to avoid the overhead of creating an intermediate file, which can be very large.

Schema of the Generated Database

The following sections describe the schema of the database created through the SQL backend.

Table IDS

Details of interdependant identifiers appearing in the workspace.

Field name Field type Value description
EIDINTEGER or BIGINT1Unique identifier key
NAMECHARACTER VARYINGIdentifier name
READONLYBOOLEANTrue if it appears in at least one read-only file
UNDEFMACROBOOLEANTrue if it is apparantly an undefined macro
MACROBOOLEANTrue if it a preprocessor macro
MACROARGBOOLEANTrue if it a preprocessor macro argument
ORDINARYBOOLEANTrue if it is an ordinary identifier (variable or function)
SUETAGBOOLEANTrue if it is a structure, union, or enumeration tag
SUMEMBERBOOLEANTrue if it is a structure or union member
LABELBOOLEANTrue if it is a label
TYPEDEFBOOLEANTrue if it is a typedef
ENUMBOOLEANTrue if it is an enumeration member
FUNBOOLEANTrue if it is a function name
CSCOPEBOOLEANTrue if its scope is a compilation unit
LSCOPEBOOLEANTrue if it has linkage scope
UNUSEDBOOLEANTrue if it is not used

Table FILES

File details.

Field name Field type Value description
FIDINTEGERUnique file key
NAMECHARACTER VARYINGFile name
ROBOOLEANTrue if the file is read-only
NCHARINTEGERNumber of characters
NCCOMMENTINTEGERNumber of comment characters
NSPACEINTEGERNumber of space characters
NLCOMMENTINTEGERNumber of line comments
NBCOMMENTINTEGERNumber of block comments
NLINEINTEGERNumber of lines
MAXLINELENINTEGERMaximum number of characters in a line
NSTRINGINTEGERNumber of character strings
NULINEINTEGERNumber of unprocessed lines
NPPTOKENINTEGERNumber of preprocessed tokens
NCTOKENINTEGERNumber of compiled tokens
NPPDIRECTIVEINTEGERNumber of C preprocessor directives
NPPCONDINTEGERNumber of processed C preprocessor conditionals (ifdef, if, elif)
NPPFMACROINTEGERNumber of defined C preprocessor function-like macros
NPPOMACROINTEGERNumber of defined C preprocessor object-like macros
NSTATEMENTINTEGERNumber of statements
NCOPIESINTEGERNumber of copies of the file
NPFUNCTIONINTEGERNumber of defined project-scope functions
NFFUNCTIONINTEGERNumber of defined file-scope (static) functions
NPVARINTEGERNumber of defined project-scope variables
NFVARINTEGERNumber of defined file-scope (static) variables
NAGGREGATEINTEGERNumber of complete aggregate (struct/union) declarations
NAMEMBERINTEGERNumber of declared aggregate (struct/union) members
NENUMINTEGERNumber of complete enumeration declarations
NEMEMBERINTEGERNumber of declared enumeration elements
NINCFILEINTEGERNumber of directly included files

Table TOKENS

Instances of identifier tokens within the source code.

Field name Field type Value description
FIDINTEGERFile key (references FILES)
FOFFSETINTEGEROffset within the file
EIDINTEGER or BIGINT1Identifier key (references IDS)

Table COMMENTS

Comments in the code.

Field name Field type Value description
FIDINTEGERFile key (references FILES)
FOFFSETINTEGEROffset within the file
COMMENTCHARACTER VARYINGThe comment, including its delimiters

Table STRINGS

Strings in the code.

Field name Field type Value description
FIDINTEGERFile key (references FILES)
FOFFSETINTEGEROffset within the file
STRINGCHARACTER VARYINGThe string, including its delimiters

Table REST

Remaining, non-identifier source code.

Field name Field type Value description
FIDINTEGERFile key (references FILES)
FOFFSETINTEGEROffset within the file
CODECHARACTER VARYINGThe actual code

Table LINEPOS

Line number offsets within each file.

Field name Field type Value description
FIDINTEGERFile key (references FILES)
FOFFSETINTEGEROffset within the file
LNUMINTEGERLine number (starts at 1)

Table PROJECTS

Project details.

Field name Field type Value description
PIDINTEGERUnique project key
NAMECHARACTER VARYINGProject name

Table IDPROJ

Identifiers appearing in projects.

Field name Field type Value description
EIDINTEGER or BIGINT1Identifier key (references IDS)
PIDINTEGERProject key (references PROJECTS)

Table FILEPROJ

Files used in projects.

Field name Field type Value description
FIDINTEGERFile key (references FILES)
PIDINTEGERProject key (references PROJECTS)

Table DEFINERS

Included files defining required elements for a given compilation unit and project.

Field name Field type Value description
PIDINTEGERProject key (references PROJECTS)
CUIDINTEGERCompilation unit key (references FILES)
BASEFILEIDINTEGERFile (often .c) requiring (using) a definition (references FILES)
DEFINERIDINTEGERFile (often .h) providing a definition (references FILES)

Table INCLUDERS

Included files including files for a given compilation unit and project.

Field name Field type Value description
PIDINTEGERProject key (references PROJECTS)
CUIDINTEGERCompilation unit key (references FILES)
BASEFILEIDINTEGERFile included in the compilation (references FILES)
INCLUDERIDINTEGERFiles that include it (references FILES)

Table PROVIDERS

Included files providing code or data for a given compilation unit and project.

Field name Field type Value description
PIDINTEGERProject key (references PROJECTS)
CUIDINTEGERCompilation unit key (references FILES)
PROVIDERIDINTEGERIncluded file (references FILES)

Table INCTRIGGERS

Tokens requiring file inclusion for a given compilation unit and project.

Field name Field type Value description
PIDINTEGERProject key (references PROJECTS)
CUIDINTEGERCompilation unit key (references FILES)
BASEFILEIDINTEGERFile requiring a definition (references FILES)
DEFINERIDINTEGERFile providing a definition (references FILES)
FOFFSETINTEGERDefinition's offset within the providing file
LENINTEGERToken's length

Table FUNCTIONS

C functions and function-like macros.

Field name Field type Value description
IDINTEGER or BIGINT1Unique function identifier
NAMECHARACTER VARYINGFunction name (redundant; see FUNCTIONID)
ISMACROBOOLEANTrue if a function-like macro (otherwise a C function)
DEFINEDBOOLEANTrue if the function is defined within the workspace
DECLAREDBOOLEANTrue if the function is declared within the workspace
FILESCOPEDBOOLEANTrue if the function's scope is a single compilation unit (static or macro)
FIDINTEGERFile key of the function's definition, declaration, or use (references FILES)
FOFFSETINTEGEROffset of definition, declaration, or use within the file
FANININTEGERFan-in (number of callers)

Table FUNCTIONMETRICS

Metrics of defined functions and macros.

Field name Field type Value description
FUNCTIONIDINTEGER or BIGINT1Function identifier key (references FUNCTIONS)
NCHARINTEGERNumber of characters
NCCOMMENTINTEGERNumber of comment characters
NSPACEINTEGERNumber of space characters
NLCOMMENTINTEGERNumber of line comments
NBCOMMENTINTEGERNumber of block comments
NLINEINTEGERNumber of lines
MAXLINELENINTEGERMaximum number of characters in a line
NSTRINGINTEGERNumber of character strings
NULINEINTEGERNumber of unprocessed lines
NPPTOKENINTEGERNumber of preprocessed tokens
NCTOKENINTEGERNumber of compiled tokens
NPPDIRECTIVEINTEGERNumber of C preprocessor directives
NPPCONDINTEGERNumber of processed C preprocessor conditionals (ifdef, if, elif)
NPPFMACROINTEGERNumber of defined C preprocessor function-like macros
NPPOMACROINTEGERNumber of defined C preprocessor object-like macros
NSTMTINTEGERNumber of statements or declarations
NOPINTEGERNumber of operators
NUOPINTEGERNumber of unique operators
NNCONSTINTEGERNumber of numeric constants
NCLITINTEGERNumber of character literals
NIFINTEGERNumber of if statements
NELSEINTEGERNumber of else clauses
NSWITCHINTEGERNumber of switch statements
NCASEINTEGERNumber of case labels
NDEFAULTINTEGERNumber of default labels
NBREAKINTEGERNumber of break statements
NFORINTEGERNumber of for statements
NWHILEINTEGERNumber of while statements
NDOINTEGERNumber of do statements
NCONTINUEINTEGERNumber of continue statements
NGOTOINTEGERNumber of goto statements
NRETURNINTEGERNumber of return statements
NPIDINTEGERNumber of project-scope identifiers
NFIDINTEGERNumber of file-scope (static) identifiers
NMIDINTEGERNumber of macro identifiers
NIDINTEGERTotal number of object and object-like identifiers
NUPIDINTEGERNumber of unique project-scope identifiers
NUFIDINTEGERNumber of unique file-scope (static) identifiers
NUMIDINTEGERNumber of unique macro identifiers
NUIDINTEGERNumber of unique object and object-like identifiers
NGNSOCINTEGERNumber of global namespace occupants at function's top
NPARAMINTEGERNumber of parameters
MAXNESTINTEGERMaximum level of statement nesting
NLABELINTEGERNumber of goto labels
FANININTEGERFan-in (number of calling functions)
FANOUTINTEGERFan-out (number of called functions)
CCYCL1INTEGERCyclomatic complexity (control statements)
CCYCL2INTEGERExtended cyclomatic complexity (includes branching operators)
CCYCL3INTEGERMaximum cyclomatic complexity (includes branching operators and all switch branches)
CSTRUCREALStructure complexity (Henry and Kafura)
CHALREALHalstead complexity
IFLOWREALInformation flow metric (Henry and Selig)
FIDBEGININTEGERFile key of the function's definition begin (references FILES)
FOFFSETBEGININTEGEROffset of definition begin within the file
FIDENDINTEGERFile key of the function's definition end (references FILES)
FOFFSETENDINTEGEROffset of definition end within the file

Table FUNCTIONID

Identifiers comprising a function's name.

Field name Field type Value description
FUNCTIONIDINTEGER or BIGINT1Function identifier key (references FUNCTIONS)
ORDINALINTEGERPosition of the identifier within the function name (0-based)
EIDINTEGER or BIGINT1Identifier key (references IDS)

Table FCALLS

Function calls.

Field name Field type Value description
SOURCEIDINTEGER or BIGINT1Calling function identifier key (references FUNCTIONS)
DESTIDINTEGER or BIGINT1Called function identifier key (references FUNCTIONS)

Table FILECOPIES

Files occuring in more than one copy.

Field name Field type Value description
GROUPIDINTEGERFile group identifier
FIDINTEGERKey of file belonging to a group of identical files (references FILES)

Note 1: INTEGER on 32-bit architectures, BIGINT on 64-bit archiectures.

Examples of SQL Queries

Once data has been uploaded onto a SQL database it can be queried in a number of different ways. Here are some example queries. Note that some queries may not run on a particular relational database engine.

Find identifiers of a given type (typedefs, in this case):

select name from
ids left join tokens on ids.eid = tokens.eid
where ids.typedef = true

Number of different files that use a given identifier:

select name, count(*) as cf from (
 select fid, tokens.eid, count(*) as c from
 tokens
 group by
 eid, fid) as cl inner join ids on
cl.eid = ids.eid
group by ids.eid, ids.name
order by cf desc;

Number of times an identifier occurs in a single file:

SELECT IDS.NAME AS INAME, FILES.NAME AS FNAME, COUNT(*) AS C FROM TOKENS
INNER JOIN IDS ON
IDS.EID = TOKENS.EID
INNER JOIN FILES ON
TOKENS.FID = FILES.FID
GROUP BY IDS.EID, TOKENS.FID
ORDER BY C DESC;

Number of times an identifier occurs in the workspace:

select name, count(*) as c from tokens
inner join ids on
ids.eid = tokens.eid
group by eid
order by c desc

Reconstitute the file with fid = 4:

select s from
(select name as s, foffset  from ids inner join tokens on
ids.eid = tokens.eid where fid = 4
union select code as s, foffset from rest where fid = 4
union select comment as s, foffset from comments where fid = 4
union select string as s, foffset from strings where fid = 4
)
order by foffset
The result will have newlines in the wrong places. Piping the output through a shell script like the following can fix this problem.
sed -e '/^[0-9][0-9]* rows/d' |
tr -d '\n' |
sed 's/\\u0000d/\
/g'
The above script will massage the HSQLDB output removing the trailing N rows line and all existing newlines, and changing the embedded \u0000d sequences into newlines. For the Windows line-end conventions the same script would be:
sed -e '/^[0-9][0-9]* rows/d' |
tr -d '\n\r' |
sed 's/\\u0000d\\u0000a/\
/g'

Show the projects each identifier belongs to:

select IDS.NAME, PROJECTS.NAME from IDS
INNER JOIN IDPROJ ON IDS.EID = IDPROJ.EID
INNER JOIN PROJECTS ON IDPROJ.PID = PROJECTS.PID
ORDER BY IDS.NAME;

Show the included files required by other files for each compilation unit and project.

select
        projects.name as projname,
        cufiles.name as cuname,
        basefiles.name as basename,
        definefiles.name as defname
from
        definers inner join projects on definers.pid = projects.pid
        inner join files as cufiles on definers.cuid=cufiles.fid
        inner join  files as basefiles on definers.basefileid=basefiles.fid
        inner join files as definefiles on definers.definerid = definefiles.fid;

Speed-up processing:

create index teid on tokens(eid)
create index tfid on tokens(fid)

Obtain identifiers common between files participating in a define/use relationship:

SELECT
    tokensa.eid,
    min(ids.name) as identifier,
    min(filesb.name) as defined,
    min(filesa.name) as used
FROM definers
INNER JOIN tokens AS tokensa ON definers.basefileid = tokensa.fid
INNER JOIN tokens AS tokensb ON definers.definerid = tokensb.fid
INNER JOIN ids ON ids.eid = tokensa.eid
INNER JOIN files as filesa ON tokensa.fid = filesa.fid
INNER JOIN files as filesb ON tokensb.fid = filesb.fid
WHERE tokensa.eid = tokensb.eid
GROUP BY tokensa.eid, definerid, basefileid
ORDER BY defined, identifier

Create a function and macro call graph:

SELECT source.name AS CallingFunction, dest.name AS CalledFunction
FROM fcalls
INNER JOIN functions AS source ON fcalls.sourceid = source.id
INNER JOIN functions AS dest ON fcalls.destid = dest.id

Details of the Collected Metrics

The following sections provide details for some of the collected function and file metrics. Although the metrics collected by CScout are considerably more accurate than those collected by programs that either do not parse the source code or parse the preprocessed code, they still employ approximations.

Metrics Common to Files and Functions

Number of C preprocessor directives
See note 1.
Number of processed C preprocessor conditionals (ifdef, if, elif)
See note 1.
Number of defined C preprocessor function-like macros
See note 1.
Number of defined C preprocessor object-like macros
See note 1.
Number of preprocessed tokens
Although during preprocessing whitespace is considered a valid token, this metric does not take whitespace tokens into account. This makes it easy to compare the number of preprocessed tokens with the number of compiled tokens. The two metrics are equal if no macro expansion takes place.
Number of compiled tokens
See note 1.

File-Specific Metrics

Number of statements
This metric measures number of statements parsed while processing the file, including statements generated by macro expansion. See note 1.
Number of defined project-scope functions
See note 1.
Number of defined file-scope (static) functions
See note 1.
Number of defined project-scope variables
See note 1.
Number of defined file-scope (static) variables
See note 1.
Number of complete aggregate (struct/union) declarations
Also includes complete declarations without a tag. See note 1.
Number of declared aggregate (struct/union) members
See note 1.
Number of complete enumeration declarations
See note 1.
Number of declared enumeration elements
See note 1.

Function-Specific Metrics

Number of statements or declarations
Nested statements are counted recursively. Thus
while (a)
        if (b)
                c();
counts as three statements.
Number of operators
See note 2.
Number of unique operators
See note 2.
Number of if statements
See note 3.
Number of else clauses
See note 3.
Number of switch statements
See note 3.
Number of case labels
See note 3.
Number of default labels
See note 3.
Number of break statements
See note 3.
Number of for statements
See note 3.
Number of while statements
This metric does not include the do .. while form. See note 3.
Number of do statements
See note 3.
Number of continue statements
See note 3.
Number of goto statements
See note 3.
Number of return statements
See note 3.
Total number of object and object-like identifiers
Also includes macros.
Number of unique object and object-like identifiers
Also includes macros.
Number of global namespace occupants at function's top
This metric measures the namespace pollution in the object namespace at the point before entering a function. Its value is the count of all macros as well as objects with file and project-scope visibility that are declared at the point it is measured. See note 1. See note 4.
Number of parameters
See note 1.
Maximum level of statement nesting
In order to avoid excessively inflating this metric when measuring sequences of the form
if (a) {
        ...
} else if (b) {
        ...
} else if (c) {
        ...
} else
        ...
}
this metric does not take into account the nesting of else clauses. Thus the above code will be given a nesting level of 1, rather than 3, which is implied by the following (actual) reading of the code.
if (a) {
        ...
} else
        if (b) {
                ...
        } else
                if (c) {
                        ...
                } else
                        ...
                }
See note 1. See note 4.
Fan-in (number of calling functions)
This is also listed under a function's details for functions that are not defined (and have not metrics associated with them).
Cyclomatic complexity (control statements)
This metric, CC1 measures the number of branch points in the function. In order to avoid misleadingly high values that occur from even trivial switch statements, this metric measures the complexity of a switch statement as 1.
Extended cyclomatic complexity (includes branching operators)
This metric, CC2, takes into account the nodes introduced by the Boolean-AND, boolean-OR, and conditional evaluation operators.
Maximum cyclomatic complexity (includes branching operators and all switch branches)
This metric, CC3, considers each case label as a separate node.
Structure complexity (Henry and Kafura)
This metric is calculcated as follows.
Cp = (fan_in * fan_out)2
Halstead complexity
This metric is calculcated as follows.
HC = (number_of_operators + number_of_operands) * log2( unique_number_of_operators + unique_number_of_operands)
Where operands are object identifiers, macros, numeric and character constants. For the purpose of determining unique operands, each numeric or character constant is considered a separate operand.
Information flow metric (Henry and Selig)
This metric is calculcated as follows.
HCp = CC1 * Cp

Notes

Note 1
This metric is measured the first time a file is encountered in a project. The metric does not take into account regions that were not processed due to conditional compilation.
Note 2
This metric is calculated before preprocessing, so as to account operators occuring in function-like macros to the corresponding macro. However, this makes it difficult to differentiate between commas used to separate function arguments and the comma operator. As a result the comma is ignored as an operator.
Note 3
This metric is calculated before preprocessing, so as to account keywords occuring in function-like macros to the corresponding macro. As a result C keywords used during preprocessing as identifiers, as in
#define x(if, while, else) (if + while + else)
will be miscounted as keywords occuring in the corresponding macro. Furthermore keywords generated during preprocessing, as in
#define WHILE(x) while(x) {
#define WEND     }

WHILE (x)
        foo();
WEND
will not be counted as occuring in the corresponding C function.
Note 4
This metric is not measured for function-like macros.

Shortcomings

The nature of the C language and its preprocessor can result in pathological cases that can confuse the CScout analysis and substitution engine. In all cases the confusion only results in erroneous analysis or substitutions of the particular identifiers and will not affect other parts of the code. In some cases you can even slightly modify your workspace definition or code to ensure CScout works as you intend. The following cases are the most important in recognising and substituting identifiers:
  1. Conditional compilation

    Some programs have parts of them compiled under conditional preprocessor directives. Consider the following example:

    #ifdef unix
    #include <unistd.h>
    #define erase_file(x) unlink(x)
    #endif
    
    #ifdef WIN32
    #include <windows.h>
    #define erase_file(x) DeleteFile(x)
    #endif
    
    main(int argc, char *argv[])
    {
            erase_file(argv[1]);
    }
    
    As humans we can understand that erase_file occurs three times within the file. However, because CScout preprocesses the file following the C preprocessor semantics, it will typically match only two instances. In some cases you can get around this problem by defining macros that will ensure that all code inside conditional directives gets processed. In other cases this will result in errors (e.g. a duplicate macro definition in the above example). In such cases you can include in your workspace the same project multiple times, each time with a different set of defined macros.
    workspace example {
    	project idtest {
    		define DEBUG 1
    		define TEST 1
    		file idtest.c util.c
    	}
    	project idtest2 {
    		define NDEBUG 1
    		define PRODUCTION
    		file idtest.c util.c
    }
    
  2. Partial coverage of macro use

    Consider the following example:

    struct s1 {
            int id;
    } a;
    
    struct s2 {
            char id;
    } b;
    
    struct s3 {
            double id;
    } c;
    
    #define getid(x) ((x)->id)
    
    main()
    {
            printf("%d %c", getid(a), getid(b));
    }
    
    In the above example, changing an id instance should also change the other three instances. However, CScout will not associate the member of s3 with the identifier appearing in the getid macro or the s1 or s2 structures, because there is no getid macro invocation to link them together. If e.g. id is replaced with val the program will compile and function correctly, but when one tries to access the c struture's member in the future using getid an error will result.
    struct s1 {
            int val;
    } a;
    
    struct s2 {
            char val;
    } b;
    
    struct s3 {
            double id;
    } c;
    
    #define getid(x) ((x)->val)
    
    main()
    {
            printf("%d %c", getid(a), getid(b));   /* OK */
            printf(" %g", getid(c));               /* New statement: error */
    }
    
    To avoid this (rare) problem you can introduce dummy macro invocations of the form:
    #ifdef CSCOUT
            (void)getid(d)
    #endif
    
  3. Undefined macros

    We employ a heuristic classifying all instances of an undefined macro as being the same identifier. Thus in the following sequence foo will match all three macro instances:

    #undef foo
    
    #ifdef foo
    #endif
    
    #ifdef foo
    #endif
    
    #define foo 1
    
    In most cases this is what you want, but there may be cases where the macro appears in different files and with a different meaning. In such cases the undefined instances of the macro will erroneously match the defined instance.

In addition, the analysis of functions can be confused by the following situations.

  1. Functions getting called through function pointers will not appear in the call graphs. This is a common limitation of static call analysis.
  2. Function-like macros called from inside function bodies that were generated by macro expansion will not be registered as calls.
  3. Non-function like macros that expand into function calls will not appear in the call graph; the corresponding functions will appear to be called by the function containing the macro.

Error Messages

Warnings

Fatal Errors

Errors

License

The free unsupported CScout version is distributed under the CScout Public License. It allows free use of CScout for analyzing and modifying free/Open Source software.

For using CScout on non-free/proprietary software, the CScout supported version and associated license are available. The supported version comes with a normal commercial software license, with none of the special restrictions of this license.


THE CSCOUT PUBLIC LICENSE version 1.0

Copyright (C) 2003 Diomidis Spinelllis, Athens, Greece.
Everyone is permitted to copy and distribute this license document.

The intent of this license is to establish freedom to use, share, and change the software regulated by this license under the open source model.

This license applies to any software containing a notice placed by the copyright holder saying that it is covered by the terms of the CSCOUT Public License version 1.0. Such software is herein referred to as the Software. This license covers modification and distribution of the Software and the use of the Software for the development and maintenance of free software.

Granted Rights

1. You are granted the non-exclusive rights set forth in this license provided you agree to and comply with any and all conditions in this license. Whole or partial distribution or use of the Software in any form or way signifies acceptance of this license.

2. You may copy and distribute the Software in unmodified form provided that the entire package, including - but not restricted to - copyright, trademark notices and disclaimers, as released by the initial developer of the Software, is distributed under this license.

3. You may make modifications to the Software's source code and distribute your modifications, in a form that is separate from the Software, such as patches. The following restrictions apply to modifications:

a. Modifications must not alter or remove any copyright notices in the Software.

b. When modifications to the Software are released under this license, a non-exclusive royalty-free right is granted to the initial developer of the Software to distribute your modification in future versions of the Software provided such versions remain available under these terms in addition to any other license(s) of the initial developer.

c. The machine-executable (compiled) parts of the Software shall not be modified.

4. You may use the original or modified versions of the Software to analyze and modify application programs, libraries, or other software legally developed by you or by others provided that when these items are distributed in any form you satisfy following requirements:

a. You must ensure that all recipients of machine-executable forms of these items are also able to receive and use the complete machine-readable source code to the items without any charge beyond the costs of data transfer.

b. You must explicitly license all recipients of your items to use and re-distribute original and modified versions of the items in both machine-executable and source code forms. The recipients must be able to do so without any charges whatsoever, and they must be able to re-distribute to anyone they choose.

c. If the items are not available to the general public, and the initial developer of the Software requests a copy of the items, then you must supply one.

5. You acknowledge and accept the fact that the Software may contain technical measures to enforce parts of this license (such as providing the public with a browsable version of the code you are analyzing, and the transmission of workspace-related data) and agree not to interfere with these measures.

Limitations of Liability

In no event shall the initial developers or copyright holders be liable for any damages whatsoever, including - but not restricted to - lost revenue or profits or other direct, indirect, special, incidental or consequential damages, even if they have been advised of the possibility of such damages, except to the extent invariable law, if any, provides otherwise.

No Warranty

THE SOFTWARE AND ITS DOCUMENTATION ARE PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF DESIGN, MERCHANTIBILITY, OR FITNESS FOR A PARTICULAR PURPOSE.

Choice of Law

This license is governed by the Laws of Greece. Disputes shall be settled by the Courts of Athens.


Differences between the unsupported free version and the supported version licence and software

Free / Unsupported Version Supported Version
Distributed under the CScout public license. Distributed under a commercial software license and a support contract.
Shall only be used on open source software. Can be used on proprietary software.
Unsupported. Includes 8 hours of email-based installation and configuration support and two years of free software updates.
After a large workspace is processed, the workspace is registered for public browsing at CScout's Web site. Project metrics are sent to the Web site and recorded for statistical processing. Web communication is only performed for validating the software's license. No details other than the host's name and IP address are communicated.
Can only be run on machines with a direct connection to the Internet. A proxy host and port can be specified for connecting to the CScout registration and licensing server.
The entire Internet is allowed read-only access to the CScout server. Access is regulated by a fully configurable access control list, defaulting to localhost-only access.
Only users on the local host are allowed read-write access to the server. Read-write access through specified remote hosts is possible.
Includes SQL back-end.
Includes C source obfuscation back-end.
Includes support for browse-only multiuser operation and an access log.

Frequently Asked Questions

Contents

How do I handle conditional compilation?

You can either define macros that will cover all conditional cases, or process the same project multiple times using different macro definitions. See this page.

How can I handle automatically generated files?

Some projects use mini domain-specific languages similar to yacc and lex to express some of their elements. CScout can natively parse C and yacc source files, but no other language. Obviously changes should be performed in the original domain-specific files, rather than the generated C code. On the other hand, CScout can not parse the original files, but can parse the generated code. To escape this situation include the automatically generated file in your workspace definition, but define it as read-only. In this way CScout will not allow you to modify identifiers appearing in it.

How can I save an identifier or file query?

Simply bookmark the page that shows the query's results. You can even pass the URL around or print it on a T-shirt; the URL contains the whole query.

Why aren't my call graphs appearing in the form I specified?

Changing the global options that specify the format of call graphs affects the types of links appearing in the corresponding pages. If you go to a previous page using the back button and you do not reload it, you will use the old links and will obtain the old type of call graph. This is the only instance where the use of the back button will surprise you.

How can I manually convert a CScout project to obtain its preprocessed output with cscout -E?

Simply replace the line #pragma process filename with #include filename.

Why is my read-only prefix pragma not working under Windows?

Filename matching under Windows is a difficult subject. Filenames retain case, but are compared in a case insensitive manner. To avoid problems, when writing ro_prefix and ipath pragmas under a Windows platform, respect the following rules.

My system appears to be using the hard disk excessively (thrashing). Why?

CScout tags and follows each and every identifier of the source code it processes, including header files. As a result, the memory requirements of CScout are considerable. Typical memory requirements are 700-1600 bytes per line processed. If your system's main memory is less than the ammount needed, CScout will page to disk and thrashing will occur.

Isn't the CScout logo infringing the intellectual property of the International Scout movement?

The emblem of the International Scout movement is based on the fleur de lys, a traditional design of the 11th century that was later used to decorate maps and compass cards. CScout acts as a compass and a map for C code, so the association with the fleur de lys is particularly relevant. You can read more about the fleur de lys symbol at the Wikipedia (http://www.wikipedia.org/wiki/Fleur-de-lis) web site entry. Places or institutions that use the symbol informally or as part of their heraldic arms are: Quebec; Canada; Augsburg, Germany; Florence, Italy; Slovenia; the Fuggers medieval banking family; Bosnia and Herzegovina; and Louisville, Kentucky; the Prince of Wales also has a fleur de lys on his coat of arms. A Google image search (http://images.google.com.gr/images?q=fleur+de+lys&hl=en&lr=&ie=ISO-8859-1&safe=on) for the image will also show you tens of similar designs. Finally, note that the International Scout movement's rendering of the image includes two stars on the left and right leaves. CScout's logo does not contain this distinctive feature.

When does CScout phone home?

CScout will contact its home base after processing 2MB of source code (including headers), or if the processing time exceeds 10s. These limits allow you to process the supplied awk source code, or experiment with files including windows.h

What details does CScout send when calling its home base?

Your name, address, and credit card number :-)

Seriously, the data sent to our server consists of

The authentication system is really a child's toy, even an idiot would be able to hack it, so don't bother.

The cscout Command Manual Page

NAME
SYNOPSIS
DESCRIPTION
OPTIONS
MORE OPTIONS
EXAMPLE
SEE ALSO
AUTHOR

NAME

cscout - C code analyzer and refactoring browser

SYNOPSIS

cscout [-bcErv3] [-l log file] [-p port] [-m specification] [-o | -s db] [-H proxy host] [-P proxy host port] [-A username:password] file

DESCRIPTION

CScout is a source code analyzer and refactoring browser for collections of C programs. It can process workspaces of multiple projects (we define a project as a collection of C source files that are linked together) mapping the complexity introduced by the C preprocessor back into the original C source code files. CScout takes advantage of modern hardware advances (fast processors and large memory capacities) to analyze C source code beyond the level of detail and accuracy provided by current compilers and linkers. The analysis CScout performs takes into account the identifier scopes introduced by the C preprocessor and the C language proper scopes and namespaces.

CScout as a source code analyzer can:

annotate source code with hyperlinks to each identifier

list files that would be affected by changing a specific identifier

determine whether a given identifier belongs to the application or to an external library based on the accessibility and location of the header files that declare or define it

locate unused identifiers taking into account inter-project dependencies

perform queries for identifiers based on their namespace, scope, reachability, and regular expressions of their name and the filename(s) they are found in,

perform queries for files, based on their metrics, or properties of the identifiers they contain

monitor and report superfluously included header files

provide accurate metrics on identifiers and files

More importantly, CScout helps you in refactoring code by identifying dead objects to remove, and can automatically perform accurate global rename identifier refactorings. CScout will automatically rename identifiers

taking into account the namespace of each identifier: a renaming of a structure tag, member, or a statement label will not affect variables with the same name

respecting the scope of the renamed identifier: a rename can affect multiple files, or variables within a single block, exactly matching the semantics the C compiler would enforce

across multiple projects when the same identifier is defined in common shared include files

occuring in macro bodies and parts of other identifiers, when these are created through the C preprocessor’s token concatenation feature

This manual page describes the CScout invocation and command-line options. Details about its web interface, setup, and configuration can be found in the online hypertext documentation and at the project’s home page http://www.spinellis.gr/cscout.

OPTIONS

-c

Exit immediately after processing the specified files. Useful, when you simply want to check the source code for errors.

-E

Preprocess the specified file and send the result to the standard output. Note that for this option to work correctly, you need to also process the workspace definition file with -E.

-p port

The web server will listen for requests on the TCP port number specified. By default the CScout server will listen at port 8081. The port number must be in the range 1024-32767.

-m specification

Specify the type of identifiers that CScout will monitor. The identifier attribute specification is given using the syntax: Y|L|E|T[:attr1][:attr2]... The meaning of the first letter is:

Y:

Match any of the specified attributes

L:

Match all of the specified attributes

E:

Exclude the specified attributes matched

T:

Exact match of the specified attributes

Allowable attribute names and their corresponding meanings are:

unused:

Unused identifier

writable:

Writable identifier

ro:

Read-only identifier

tag:

Tag for a struct/union/enum

member:

Member of a struct/union

label:

Label

obj:

Ordinary identifier (note that enumeration constants and typedefs belong to the ordinary identifier namespace)

macro:

Preprocessor macro

umacro:

Undefined preprocessor macro

macroarg:

Preprocessor macro argument

fscope:

Identifier with file scope

pscope:

Identifier with project scope

typedef:

Typedef

enumconst:

Enumeration constant

The -m flag can provide enormous savings on the memory CScout uses (specify e.g. -m Y:pscope to only track project-global identifiers), but the processing CScout performs under this flag is unsound. The flag should therefore be used only if you are running short of memory. There are cases where the use of preprocessor macros can change the attributes of a given identifier shared between different files. Since the -m optimization is performed after each single file is processed, the locations where an identifier is found may be misrepresented.

-r

Report on the standard error output warnings about unused and wrongly scoped identifiers and unused included files. The error message format is compatible with gcc and can therefore be automatically processed by editors that recognize this format.

-v

Display the CScout version and copyright information and exit.

-3

Implement support for trigraph characters.

MORE OPTIONS

The following options are only available on the version of CScout available through a support license.

-b

Operate in multiuser browse-only mode. In this mode the web server can concurrently process multiple requests. All web operations that can affect the server’s functioning (such as setting the various options, renaming identifiers, refactoring function arguments, selecting a project, editing a file, or terminating the server) are prohibited. Call graphs are truncated to 1000 elements (nodes or edges).

-H proxy host

Specify a proxy HTTP host for connecting to the program’s auditing and licensing server.

-P proxy host port

Specify the proxy HTTP host port for connecting to the program’s auditing and licensing server. The default port is 80.

-s database dialect

Dump the workspace contents as an SQL script. Specify help as the database dialect to obtain a list of supported database back-ends.

-A username:password

Specify a proxy host authorization username and password for connecting to the program’s auditing and licensing server.

-l log file

Specify the location of a file where web requests will be logged.

-o

Create obfuscated versions of all the writable files of the workspace.

EXAMPLE

Assume you want to analyze three programs in /usr/src/bin. You first create the following project definition file, bin.prj.

# Some small tools from the src/bin directory
workspace bin {
        ro_prefix "/usr/include"
        cd "/usr/src/bin"
        project cp {
                cd "cp"
                file cp.c utils.c
        }
        project echo {
                cd "echo"
                file echo.c
        }
        project date {
                cd "date"
                file date.c
        }
}

Then you compile the workspace file bin.prj by running the CScout workspace compiler cswc on it, and finally you run cscout on the compiled workspace file. At that point you are ready to analyze your code and rename its identifiers through your web browser.

$ cswc bin.prj >bin.cs
$ cscout bin.cs
Processing workspace bin
Entering directory /usr/src/bin
Processing project cp
Entering directory cp
Processing file cp.c
Done processing file cp.c
Processing file utils.c
Done processing file utils.c
Exiting directory cp
Done processing project cp
Processing project echo
Entering directory echo
Processing file echo.c
Done processing file echo.c
Exiting directory echo
Done processing project echo
Processing project date
Entering directory date
Processing file date.c
Done processing file date.c
Exiting directory date
Done processing project date
Exiting directory /usr/src/bin
Done processing workspace bin
Post-processing /usr/home/dds/src/cscout/bin.c
[...]
Post-processing /vol/src/bin/cp/cp.c
Post-processing /vol/src/bin/cp/extern.h
Post-processing /vol/src/bin/cp/utils.c
Post-processing /vol/src/bin/date/date.c
Post-processing /vol/src/bin/date/extern.h
Post-processing /vol/src/bin/date/vary.h
Post-processing /vol/src/bin/echo/echo.c
Processing identifiers
100%
We are now ready to serve you at http://localhost:8081

SEE ALSO

cswc(1)

AUTHOR

(C) Copyright 2003-2005 Diomidis Spinellis.


The cswc Command Manual Page

NAME
SYNOPSIS
DESCRIPTION
OPTIONS
EXAMPLE
SEE ALSO
AUTHOR

NAME

cswc - CScout workspace compiler

SYNOPSIS

cswc [-vE] [-d directory] [file]

DESCRIPTION

cswc is a workspace compiler for the CScout C source code analyzer and refactoring browser. CScout integrates in a single process the functionality of a multi-project build engine, an ANSI C preprocessor, and the parts of a C compiler up to and including the semantic analysis based on types. The build engine functionality is required to allow the user to process multiple compilation and link units as a single batch. Only thus can CScout detect dependencies across different files and projects. Each compilation unit can reside in a different directory and can require processing using different macro definitions or a different include file path. In a normal build process these options are typically specified in a Makefile. The CScout operation is similarly guided by a declarative workspace definition file. To decouple the complexity of the CScout workspace processing specification from its actual operation, and to encouriage experimentation with alternative (e.g. IDE-based) workspace specification mechanisms, CScout is guided by a very simple imperative script typically generated from more sophisticated workspace definitions by cswc, the CScout workspace compiler.

This manual page describes the cswc invocation and command-line options. Details about its input and output formats, setup, and configuration can be found in the online hypertext documentation and at the project’s home page http://www.spinellis.gr/cscout.

OPTIONS

-E

Generate a modified CScout script that will be used by CScout to preprocess the specified file and send the result to the standard output. Note that for this option to work correctly, you need to also specify -E in the CScout invocation.

-d directory

Specify the directory to use for locating the CScout configuration files.

-v

Display the cswc version and copyright information and exit.

EXAMPLE

The following is a configuration file used for processing the apache web server.

workspace apache {
    cd "/usr/local/src/apache/src"

   ro_prefix "/usr/local/src/apache/src/include/ap_config"

   # Global project definitions
    define HTTPD_ROOT "/usr/local/apache"
    define SUEXEC_BIN "/usr/local/apache/bin/suexec"
    define SHARED_CORE_DIR "/usr/local/apache/libexec"
    define DEFAULT_PIDLOG "logs/httpd.pid"
    define DEFAULT_SCOREBOARD "logs/httpd.scoreboard"
    define DEFAULT_LOCKFILE "logs/httpd.lock"
    define DEFAULT_XFERLOG "logs/access_log"
    define DEFAULT_ERRORLOG "logs/error_log"
    define TYPES_CONFIG_FILE "conf/mime.types"
    define SERVER_CONFIG_FILE "conf/httpd.conf"
    define ACCESS_CONFIG_FILE "conf/access.conf"
    define RESOURCE_CONFIG_FILE "conf/srm.conf"

   define AUX_CFLAGS
    define LINUX 22
    define USE_HSREGEX
    define NO_DL_NEEDED

   # Give project-specific directory and include path properties
    project gen_uri_delims {
        cd "main"
        ipath "../os/unix"
        ipath "../include"
        file gen_uri_delims.c
    }

   # Alternative formulation; specify per-file properties
    project gen_test_char {
        file gen_test_char.c {
            cd "main"
            ipath "../os/unix"
            ipath "../include"
        }
    }

   # httpd executable; specify directory-based properties
    project httpd {
        directory main {
            ipath "../os/unix"
            ipath "../include"
            file alloc.c buff.c http_config.c http_core.c
            file http_log.c http_main.c http_protocol.c
            file http_request.c http_vhost.c util.c util_date.c
            file util_script.c util_uri.c util_md5.c rfc1413.c
        }
        directory regex {
            ipath "."
            ipath "../os/unix"
            ipath "../include"
            define POSIX_MISTAKE
            file regcomp.c regexec.c regerror.c regfree.c
        }
        directory os/unix {
            ipath "../../os/unix"
            ipath "../../include"
            file os.c os-inline.c
        }
        directory ap {
            ipath "../os/unix"
            ipath "../include"
            file ap_cpystrn.c ap_execve.c ap_fnmatch.c ap_getpass.c
            file ap_md5c.c ap_signal.c ap_slack.c ap_snprintf.c
            file ap_sha1.c ap_checkpass.c ap_base64.c ap_ebcdic.c
        }
        directory modules/standard {
            ipath "../../os/unix"
            ipath "../../include"
            file mod_env.c mod_log_config.c mod_mime.c
            file mod_negotiation.c mod_status.c mod_include.c
            file mod_autoindex.c mod_dir.c mod_cgi.c mod_asis.c
            file mod_imap.c mod_actions.c mod_userdir.c
            file mod_alias.c mod_access.c mod_auth.c mod_setenvif.c
        }
        directory . {
            ipath "./os/unix"
            ipath "./include"
            file modules.c buildmark.c
        }
    }
}

SEE ALSO

cscout(1)

AUTHOR

(C) Copyright 2003 Diomidis Spinellis.


Bibliography

Change History

Version 2.5
(Under development.)
Version 2.4
Version 2.3
Version 2.2
Version 2.1
Version 2.0
Version 1.16
Version 1.15
Version 1.14
Version 1.13
Version 1.12
Version 1.10
Version 1.9
Version 1.8
First public release