[Top] [Prev] [Next] [Index] [TOC]

Chapter 5

ATAC: Instrumenting Your Software

Instrumentation of the software to be tested is performed by executing the compile and link commands with ATAC. ATAC analyzes and instruments code and submits it to the C or C++ compiler. This chapter discusses how to instrument the software under test with ATAC. UNIX users should read Section 5.1, Instrumenting on UNIX, Windows users should proceed to Section 5.2, Instrumenting on Windows

5.1 Instrumenting on UNIX

5.1.1 Basic Instrumentation

To compile with ATAC, remove all previously created object files and executable files you intend to build. Then compile the software as you normally would, but prefix the compile command with atac. For example, to compile an executable called wordcount from two source files wc.c and main.c, enter the following: 

	prompt:> atac cc -o wordcount wc.c main.c
In addition to creating the executable program wordcount and the object files wc.o and main.o, ATAC has created the files wc.atac and main.atac. ATAC creates a .atac file for each .c file it compiles. These files contain static coverage information used in test analysis. It is possible to restrict instrumentation and the collection of coverage information to selected source files (see Section 5.1.3, Selectively Instrumenting Software).

5.1.2 Integrating with Makefiles

If your existing Makefiles make use of the CC macro, it is easy to integrate the use of ATAC with make. When invoking make to build a target, simply redefine CC to be ATAC, as follows: 

	prompt:> make CC="atac cc"
In the case of the wordcount program, the output for a typical Makefile might look like this: 
        atac cc -c wc.c
	atac cc -c main.c
	atac cc -o wordcount wc.o main.o

5.1.3 Selectively Instrumenting Software

If you only need coverage measures for a portion of your software contained in selected source files, it is unnecessary to compile the remaining source files with ATAC. Because compilation with ATAC increases execution time, memory use, and object code size, (see Section 3.5, What Will Using ATAC Cost You?) it may be advantageous to use ATAC only on source files for which coverage measures are required.

You can limit coverage analysis of your software by selectively instrumenting specific source files. Only those source files that have been compiled with ATAC are instrumented, and only instrumented portions of the software collect trace information during test execution. This is accomplished through separate compilation, which ATAC supports in the same manner as cc. Simply compile all source files to be instrumented with atac cc and all remaining source files with the standard C compiler, then link the object files. Linking must be performed by atac cc so that the proper run-time library is included in the executable. For example, we can manually instrument wc.c and leave main.c untouched by entering: 

        prompt:> rm *.o wordcount
	prompt:> atac cc -c wc.c
	prompt:> cc -c main.c
	prompt:> atac cc -o wordcount wc.o main.o
Alternatively, we can accomplish the same thing using the existing Makefile by doing the following: 
        prompt:> make clean
	prompt:> make main.o
	prompt:> make CC="atac cc"
Note that it is not possible to selectively instrument functions within a given source file, except as described in Section 6.7, Improving Execution Speed and Saving Disk Space.

At times you may lose track of which object files or executable files were compiled with ATAC. The atacid command may be used to identify files compiled with ATAC. For example, do the following for the wordcount program: 

	prompt:> atacid wc.o main.o wordcount | fgrep ATAC
If none of the files were compiled with ATAC, no lines are printed.

5.1.4 Linking with ld

The atac cc command invokes ld, the standard UNIX linker, to link the object modules comprising an executable. If desired, you may explicitly invoke atac ld to customize this linking step. In this way, object files created with atac cc may be linked with object files and libraries created by other means.

The command line options for atac ld are the same as for ld. A program linked in this way outputs trace information to the file a.out.trace, where a.out is the name of the executable file created by atac ld. If the -o prog option is used to explicitly name the resulting executable, then the trace file generated is named prog.trace. To compile ATAC-specific object files prior to its link stage, atac ld invokes cc. After this ATAC-specific preprocessing has been completed, atac ld then invokes ld to link all object files. The use of another C compiler may be substituted for the cc command, and the use of another linker may be substituted for ld.

The following commands compile wc.c using atac cc and main.c using cc. The resulting object files are then linked to build wordcount: 

        prompt:> atac cc -c wc.c
	prompt:> cc -c main.c
	prompt:> atac ld -o wordcount /usr/lib/crt0.o wc.o main.o -lc
It is also possible to generate an ATAC instrumented executable without using a compiler to do the link.

The explanation is most easily understood by an example. Let us use the same wordcount program as used before for the illustration. To begin, create a directory, cd to it and copy the contents of the directory in which the tutorial files are installed into the new directory. There should be two c files: main.c and wc.c, and a Makefile. You can compile the wordcount program with atac by entering: 

	prompt:> make CC="atac cc"
This is the same as: 
        prompt:> atac cc  -g  -c  wc.c
	prompt:> atac cc  -g  -c   main.c
	prompt:> atac cc  -g  -o  wordcount wc.o main.o
However, you are not required to use cc to make the link. Instead of running the third command above, you can use a link editor such as ld to generate the executable. To do so, you must first run: 

	prompt:> atac_env_create
to create an atac environment file (atac_env.o) in the same directory where atac_env_create is invoked. Then you need to provide the standard C library (-lc), or any customized C libraries which are essential for your programs. Note that the exact order of these supporting libraries may vary depending on your environment. In addition, you need to supply the ATAC run time routine (atac_rt.o) in order for the executable to be able to collect the execution trace. Given all these, the command you should use looks like: 
        prompt:> ld -o wordcount wc.o main.o `ataclib`/atac_rt.o 
                    atac_env.o /usr/lib/crt0.o -lc
After a test execution, the trace information is saved in a.out.trace instead of wordcount.trace unless ATAC_TRACE is set otherwise.

5.1.5 Suppressing Instrumentation of Include Files

In C and C++ programs, source files included with the #include directive may contain executable code. By default, ATAC instruments executable code in include files unless the include file came from a system header file. The default definition of a system header file, in this context, is any include file referenced with a complete path name (starting with a /) or found in the include search list in a directory with a complete path name whose path name does not start with the user's home directory.

Include files that have been instrumented with ATAC will appear in the coverage summary by source file display. Suppressing instrumentation of selected include files may be desirable either to improve run-time performance or to remove those files from the coverage summary by source file. To modify the definition of a system file, edit the file called $HOME/.atac/cc.ini, where cc is the compile command. If this file does not exist, copy it from `ataclib`/init/cc.ini, where cc is the compile command. (To make an installation change, modify the file in `ataclib`/ini directly.) If this file does not exist either, copy from `ataclib`/init/comp.ini, where comp is the name of any similar compiler.

There should be a line in this file that begins with ``INCLUDE='' (If not, add one). The value of the INCLUDE parameter is a space separated list of include path prefixes where each path prefix is preceded by -I, -J, or -S. Source files beginning with a prefix preceded by -I, as well as source files not beginning with any prefix on the list, are treated as user files and are instrumented. Source files beginning with -J or -S are treated as system files and are not instrumented. Source files beginning with a prefix preceded by -J are treated as C include files included into C++ (appropriate extern ``C'' code is inserted before compilation).

Modify the INCLUDE parameter to specify -S for the path or path prefix for include files that you do not want instrumented.

For example:

To augment the initial definition of a system include file to exclude files from directories
/usr/nosho and mynosho, and to exclude any file in the current directory starting with Z you might use: 

INCLUDE="-S/usr/nosho/ -S./mynosho/ -S./Z -I$HOME -J/usr/include -S/"

It is important to put the -S for files under your home directory before the -I$HOME entry; otherwise the files would match -I$HOME first and would not be considered system files.

5.2 Instrumenting on Windows

5.2.1 Basic Instrumentation

To compile with ATAC, remove all previously created object files and executable files you intend to build. Then compile the software as you normally would, prefixing the compile command with ATAC. These commands are:

If you use the IBM C compiler: 

	prompt:> atacICC /Fewordcount.exe main.c wc.c
If you use the Microsoft C compiler: 

	prompt:> atacCL /Fewordcount.exe main.c wc.c
In addition to creating the executable program wordcount.exe and the object files wc.obj and main.obj, ATAC has created the files wc.atac and main.atac. ATAC creates a .atac file for each .c file it compiles. These files contain static coverage information used in test analysis. It is possible to restrict instrumentation and the collection of coverage information to selected source files (see Section 5.2.3, Selectively Instrumenting Software).

5.2.2 Integrating with Makefiles

If your existing makefiles make use of the a macro specifying a C or C++ compiler, it is easy to integrate the use of ATAC with nmake. Execute the command in Appendix A-10, Compiling with atac - IBM compiler if you are using the IBM compiler or Appendix A- 11, Compiling with atac - Microsoft compiler if you are using the Microsoft compiler. The output associated with these commands is included.

5.2.3 Selectively Instrumenting Software

If you only need coverage measures for a portion of your software contained in selected source files, it is unnecessary to compile the remaining source files with ATAC. Because compilation with ATAC increases execution time, memory use, and object code size, (see Section 3.5, What Will Using ATAC Cost You?) it may be advantageous to use ATAC only on source files for which coverage measures are required.

You can limit coverage analysis of your software by selectively instrumenting specific source files. Only those source files that have been compiled with ATAC are instrumented, and only instrumented portions of the software collect trace information during test execution. This is accomplished through separate compilation with the installed compiler. Simply compile all source files to be instrumented by prefixing your compiler with atac and all remaining source files with your installed C compiler, then link the object files. Linking must be performed by atacICC or atacCL so that the proper run-time library is included in the executable. For example, we can manually instrument wc.c and leave main.c untouched by entering:

If you use the IBM C compiler: 

        prompt:> del *.obj wordcount.exe
	prompt:> atacICC /c wc.c
	prompt:> icc /c main.c
	prompt:> atacICC /Fewordcount.exe wc.obj main.obj
If you use the Microsoft C compiler: 
        prompt:> del *.obj wordcount.exe
	prompt:> atacCL /c wc.c
	prompt:> cl /c main.c
	prompt:> atacCL /Fewordcount.exe wc.obj main.obj
Alternatively, we can accomplish the same thing using the existing makefiles by first executing the clean command for your compiler, as found in Appendix A, Platform Specific Information, then executing the following two commands, as appropriate for your compiler:

If you use the IBM C compiler: 

        prompt:> nmake -f makefile_ibm main.obj
	prompt:> nmake -f makefile_ibm CC=atacICC wordcount.exe

If you use the Microsoft C compiler: 

        prompt:> nmake -f makefile_msc main.obj
	prompt:> nmake -f makefile_msc CC=atacCL wordcount.exe
Note that it is not possible to selectively instrument functions within a given source file, except as described in Section 6.7, Improving Execution Speed and Saving Disk Space.

5.2.4 Building Executables with Installed Linkers

It is possible to generate an ATAC instrumented executable without using a compiler command to do the link.

The explanation is most easily understood by an example. Let us use the same wordcount program as used before for the illustration. To copy these files, create a new directory, cd into it, and copy the contents of the directory in which the tutorial files are installed into the new directory. There should be two c files: main.c and wc.c, and a makefile. You can compile the wordcount program with atac by executing the nmake command for your setup, as specified in Appendix A, Platform Specific Information.

This is the same as:

If you use the IBM C compiler: 

        prompt:> atacICC /c wc.c main.c
	prompt:> link /Fewordcount.exe wc.obj main.obj <ataclib>\dllatacrt.lib
If you use the Microsoft C compiler: 
        prompt:> atacCL /c wc.c main.c
	prompt:> link /Fewordcount.exe wc.obj main.obj <ataclib>\dllatacrt.lib
where <ataclib> represents the directory printed by the atac_lib command.

5.2.5 Suppressing Instrumentation of Include Files

In C and C++ programs, source files included with the #include directive may contain executable code. By default atac instruments executable code in include files unless the include file comes from a list of partial pathnames contained in the registry. The default list suppresses header files whose pathnames begin with a disk drive identifier that contains the current compiler. For example if a compiler is installed in C:\BMCPPW, then any header files beginning with C: in #include statements will not be instrumented.

Include files that have been instrumented with atac will appear in the coverage summary by source file display.

Suppressing instrumentation of selected include files may be desirable either to improve run-time performance or to remove those files from the coverage summary by source file. The xconfig utility contains an entry named NO_INSTRUMENT that may be modified at any time. The NO_INSTRUMENT list is a series of partial path names separated by semicolons. Any pathname of a header file that begins with one of the partial pathnames in the NO_INSTRUMENT list is not instrumented. Pathnames to suppress are selected by case-sensitive string matching so upper and lower case entries for the same header file name may be necessary.

5.3 Common Instrumentation Options

5.3.1 Code Inside Macros

When uncovered testable attributes exist inside a preprocessor macro expansion, ATAC display components highlight the macro name and arguments within the original source code. Normally, blocks other than the first block inside a macro expansion are not counted or displayed. It is assumed that the tester does not expect complete coverage of each macro at each invocation. These blocks may be counted using the atac -J option.

5.3.2 Marking Code for Selective Reporting

In a program there is often code that is not normally intended to be executed because it handles unexpected error conditions or it implements unsupported features or debugging options, etc. By default, ATAC includes this code in coverage displays. If the programmer does not intend to test this code, coverage displays that include this code may be distracting and, possibly, misleading.

The programmer may identify this code by inserting a NOTTESTED comment in the code indicating that it is not to be included in coverage computations. The comment should also indicate the reason the code does not need to be tested. The recommended format of the comment is:

/* NOTTESTED: reason */

where reason is one of:

Note that code that tests for user errors, input errors, or common system errors (e.g. write failed due to disk full) should be tested and should not be identified by the NOTTESTED comment. In general, code should be marked by the NOTTESTED comment only if an error in that code will not be considered an error in the system either because the code will never be executed, or the code will only be executed in a situation where failure is acceptable.

In addition to the NOTTESTED comment, ATAC supports the NOTREACHED comment used by the lint program. The NOTREACHED comment is intended to be used for code that cannot be reached in the flow of the program, e.g. code following a return statement.

The NOTTESTED comment may be inserted anywhere in the code that a comment is allowed. ATAC will not report coverage for the block containing the comment (or the next block if the comment is outside any block) nor for any following block that can only be reached by passing through that block. If the NOTTESTED comment appears outside of any subroutine, ATAC will not report coverage for the subroutine following the comment.

For example in the program dEcho.c: 

 	/* dEcho program */
	int dFlag, dCount;
	/* NOTTESTED:debug */ 
	dBug()
	{  if (dFlag) 
	      printf("D: %d\n", dCount); 
	} 
	main(int argc, char *argv[]) 
	{  int i; 
	   for (i = 1; i < argc; ++i) { 
	      ++dCount; 
	      if (!strcmp(argv[i], "-d")) { /* NOTTESTED:debug */
	             dFlag = 1; 
	         } 
	      else printf("%s ", argv[i]);
	   }
	   putchar(`\n'); 
	   dBug(); 
	   return 0; 
	}
Coverage is not reported for the entire dBug function nor for the dFlag = 1 statement.

The -T option of the atac command may be used to include code marked NOTTESTED in the coverage computation. The -R option may be used for code marked NOTREACHED. For example the block coverage on the program above before any tests are run is given by: 

        prompt:> atac -s -mb dEcho.atac 

        % blocks 
        ---------
        0(0/8)         == total ==
When the -T option is given, blocks marked NOTREACHED are included: 
	prompt:> atac -s -mbT dEcho.atac
The -mT option of the atac command may be used to include a separate column for coverage on code marked NOTTESTED. The -mR option of the atac command may be used for code marked NOTREACHED. For example the NOTREACHED, NOTTESTED, and other block coverage on the program above before any tests are run is given by: 

	prompt:> atac -s -mRTb dEcho.atac 

        % NOTREACHED    % NOTTESTED     % blocks
	-------------   ------------    ---------
	100(0)          0(0/4)          0(0/8)     == total ==
The same options may be used to view covered or non-covered code marked NOTTESTED or NOTREACHED.

These commands may be useful for determining how frequently the NOTTESTED and NOTREACHED comments have been used, and whether code was executed that was not expected to be tested.

5.4 Compilation and Link Errors

When a program fails to compile or link with ATAC, check that it compiles and links without errors using your installed compiler (Refer to Appendix A, Platform Specific Information for the specific commands to execute.) If the same command line arguments are used, a program that compiles and links without errors using a standard compiler should compile and link without errors under ATAC, with the following exceptions:

In most of these cases, ATAC will issue a parse failed message and fail to compile the program.

After preprocessing and data-flow instrumentation of a source program, ATAC passes the modified program to the standard C compiler. Errors from the standard C compiler may indicate an error in ATAC (e.g., during instrumentation) or in the standard C compiler.

A program compiled with ATAC must be linked with ATAC. If linked with the standard compiler, the linker will issue a message indicating an undefined external symbol named aTaC43 or _aTaC43 because the required run-time library has not been included in the executable.


1 American National Standard for Information Systems - Programming Language C, Document Number X3J11.

[Top] [Prev] [Next] [Index] [TOC]