17.5. make and Makefiles

In this section we discuss the make command and Makefiles. We present some examples starting from simple Makefile and building up to a more complex and more generic Makefile (Section 17.5.10) that is a good starting point for most uses. For more information about writing Makefiles see the GNU Make Manual.

17.5.1. About make

The Unix make utility simplifies compiling programs. A user (or a program) writes the Makefile with the rules to execute. A user just runs make and the make program reads in rules from the Makefile that tell it how to compile a program.

One benefit of make is that it can execute complex commands to build executable files simply by typing make versus the user having to type in long gcc commands to build each time. Larger programs are often split into separate modules consisting of multiple .c and .h files. They may also link in multiple libraries, or store header and library files in non-standard locations. All of these make compiling more complicated, and gcc command lines longer and more complex. In these cases, manually typing the full gcc command line at the shell prompt is both time consuming and error prone. For example, here is a long gcc command line that would be tedious to type by hand:

gcc -Wall -g  -I../includes -I. -o mycc emitter.c init.c lexer.c symbol.c parser.c ast.c error.c symbol_table.c codegen.c emitcode.c main.c -L../libs -lmylib -lpthread

With make the user simply types make to execute the commands listed in the Makefile.

Another benefit of make is that it only rebuilds those things that need to be rebuilt --- just the object and executable files that depend on files that have been modified since the last time they were built. For large programs, compiling everything from scratch can be very time consuming, and thus make can save a lot of compilation time by only rebuilding those things that need to be rebuilt.

A user typically writes the Makefile with the rules for make to execute. There are also programs, like CMake and GNU Autotools, that generate Makefiles. These are typically used for building large software and for easily supporting compilation across different systems that includes finding needed compilers and other software, libraries, and header files on a particular system.

17.5.2. Using make

The steps for using make are:

  1. Create a Makefile listing the rules for building the executable. By default the file should be named 'Makefile'. This only has to be done once, except when new modules (.c or .h files) are added to the program, then the Makefile needs to be updated to add new module dependencies to existing rules or to add new rules to include them in compilation.

  2. After editing program file(s), rebuild the executable by typing make:

    $ make

    A specific rule in the Makefile can be executed by typing:

    $ make target_label

    For example, to execute the rm command to remove all built files in the example Makefile listed below, type:

    $ make clean

17.5.3. Creating a Makefile

A Makefile typically starts with some variable definitions which are then followed by a set of rules (or target entries) for building specific targets (typically for building .o and executable files). Each rule consists of a target label and a set of associated commands (sometimes called recipes) that are run for the target. The following is the general format of a rule:

# comment
# (note: the <tab> in the command line is necessary for make to work)
target:  dependency1 dependency2 ...
      <tab> command

Here is an example:

# target entry to build program executable from program.o
# and util.o object files
#
program: program.o util.o
	gcc -o program program.o util.o
Make and whitespace characters

make is very picky about whitespace characters. For example, the <tab> on the line before the command part of a Makefile target entry is very important for make to correctly parse the target entry and execute the command part. If, for example, you use space characters in place of the <tab>, then make will not correctly parse the target entry and will not run its associated command.

17.5.4. A Simple Makefile

Here is an example simple Makefile that builds an executable from a single .c file named myprog.c:

# build an executable named myprog from myprog.c
all: myprog.c
	gcc -g -Wall -o myprog myprog.c

clean:
	rm myprog

Typing make triggers the command associated with the all target label to run gcc -g -Wall -o myprog myprog.c if the modification date on the file myprog.c is newer than the modification date on the file myprog or if myprog doesn’t exist. Note that the name all isn’t special, by default make picks the first target label in the Makefile to execute. To execute a specific rule other than the first one listed in the makefile, list the target name on the make command line. For example, make clean runs the rm myprog command to remove the previously built executable file myprog.

In Section 17.5.8 we show how to use Makefile variables to write more generic versions of this Makefile—​one that can be copied and minimally edited to create a new Makefile for building a different program.

17.5.5. A Makefile for multiple source files

Below is an example Makefile for building an executable from multiple .c and .h files. It first compiles each .c into a .o file, and then links the .o and libraries together to build the executable. Because it lists dependencies for each .o file, if one .c file is modified only one .o needs to be rebuilt (the one that is dependent on the modified .c) and it does not rebuild all .o files. This is a particularly nice feature that speeds up compile time for larger programs consisting of many .c files.

# Example Makefile for a program with many .c modules
# make: build count executable program
# make clean: clean-up all built files

all: count

# To create the executable file count we need the object files
# countwords.o, counter.o, and scanner.o:
count: countwords.o counter.o scanner.o
	gcc -g -Wall -o count countwords.o counter.o scanner.o

# To create the object file countwords.o, we need the source
# files countwords.c, scanner.h, and counter.h:
countwords.o: countwords.c scanner.h counter.h
	gcc -g -Wall -c countwords.c

# To create the object file counter.o, we need the source files
# counter.c and counter.h:
counter.o: counter.c counter.h
	gcc -g -Wall -c counter.c

# To create the object file scanner.o, we need the source files
# scanner.c and scanner.h:
scanner.o: scanner.c scanner.h
	gcc -g -Wall -c scanner.c

# To start over from scratch, type 'make clean'.  This
# removes the executable file, as well as old .o object
# files and *~ backup files:
clean:
	rm -f count *.o *~

After running make the first time, if the counter.c file is modified, then on the next run of make, only these two rules are executed (due to their dependency on counter.c or counter.o):

counter.o: counter.c counter.h
	gcc -g -Wall -c counter.c

count: countwords.o counter.o scanner.o
	gcc -g -Wall -o count countwords.o counter.o scanner.o

In Section 17.5.9 and Section 17.5.10 we show how to use Makefile variables, implicit Makefile rules, and automatic dependency generation, to write more generic versions of a Makefile. The Makefile in Section 17.5.10 is a good generic Makefile for applications built from many .c and .h files, and that may build with header files or libraries stored in non-standard locations.

17.5.6. make Errors

Errors in make generally stem from one of two causes. The first are errors in the syntax of the Makefile itself, and the second are compilation errors either due to missing rules in the Makefile, or due to errors in the program itself. When using make it can sometimes be difficult to determine if you need to fix the Makefile or your program.

Errors in Makefile syntax will be reported with the line number in the Makefile of the error to help you track down the syntax problem. However, there are some common errors that are difficult to see because they are related to errors with use of whitespace characters. Two example of this type are:

  • Using space characters instead of a <tab> at the start of the command part of a target entry. For example, if there are space characters at the beginning of this target entry’s second line instead of a <tab> character, make cannot correctly parse this rule:

    all: myprog.c
     $(CC) $(CFLAGS) $(INCLUDES) -o myprog myprog.c

  • Having extra spaces at the end of the prefix of a file name that the Makefile expands in other places by adding different suffixes to. For example:

    TARGET = myprog
SRC = $(TARGET).c

    If there is trailing whitespace after myprog, then instead of SRC being myprog.c it will be something like myprog .c, which is not a valid file name.

If the error is with compiling or linking, it could be due to errors in your Makefile or due to an error in your program. Some examples Makefile errors that cause compilation or linking errors include: a missing rule to compile a .o; missing dependencies not forcing re-compilation; missing explicitly linking in a needed library; or missing including paths (-I or -L) to header file and library locations needed to compile the program. Many of these errors will show up as gcc errors, but require fixes to the Makefile instead of the program source.

Some tips to help you determine if the Makefile, the source code, or both need to be fixed include:

  • As a first step to see if you are missing a dependency, run make clean and then make to rebuild everything from scratch. If it is successful, a dependency is missing in the Makefile.

  • If you don’t see a .o file that you need to link in, the problem is that you need to add a rule to build it (perhaps updating your SRCS definition to include this new file).

  • If there is an error with the compiler not being able to find a .h file, then this indicates that you need to update your INCLUDES variable to add a of the form -I[path to include directory] to tell make where to find them.

  • Similarly, if there are errors with the compiler not being able to find libraries (these show up as undefined errors during the final compilation stage, linking), you need to update your LIBDIR or LIBS definitions to include paths to libraries (-L) or to add to the command line any missing libraries that need to be linked in (-l).

  • In some cases when the error is in a C source file, such as missing #include, both the C source file and the Makefile may need to be updated to fix the error. For example, adding an #include to a C source file, may also result in updating the include (-I) path, and in some cases also a library path (-L) and explicitly linking in a library (-l) in the Makefile.

17.5.7. Parallel make

For large programs consisting of many .c and .h files, compiling can take a long time. Much of the time can be due to compiling a .o file from each .c file. Compiling each .o is independent of compiling other .o files, and thus this is a good target for parallelization where multiple threads or processes can simultaneously compile a different .c file in parallel. Using the -j[num] command line option to make tells it to run up to num jobs at a time, or to in parallel simultaneously execute num different recipes.

For example, to parallelize make to perform 12 recipes at a time type:

make -j12

On a machine with 12 or more cores, this will run about twelve times faster than make.

17.5.8. Advanced: A Generic Simple Makefile

We can rewrite the simple Makefile from Section 17.5.4 to make it more generic. A generic Makefile is one that is easier to edit for use for another program (typically users start a new Makefile with a copy of an existing one that they then modify versus writing one from scratch each time). This more generic Makefile replaces the specific application file names in rules with Makefile variables resulting in the rule being generic for any program. The syntax for defining Makefile variables is <variable_name> = <value>. For example:

# define CC to be the name of the compiler
CC = gcc

Once defined, the Makefile variable can be used later in the Makefile using $(variable_name) syntax. For example:

# use the value of CC in a target entry:
all:
    $(CC) -o myprog myprog.c

Below is an example of a slightly more generic version of the simple Makefile above to build an executable from a single .c file. This more generic version of the simple Makefile specifies variables to indicate which compiler to use (CC), the compiler flags to pass (CFLAGS, the name of the .c source file (SRCS), and the name of the program executable to build (TARGET). The RM variable is a builtin Makefile variable that is defined to be the rm command. You can change the value of any builtin variables by redefining them to be something else in the Makefile.

By using Makefile variables, the result is that everything in the Makefile starting with all: is generic and will work for any program. Only the SRCS and TARGET variables need to be changed to build a different program.

# define the compiler: gcc for C programs
CC = gcc

# define the compiler flags:
#  -g    adds debugging information to the executable file
#  -Wall turns on most, but not all, compiler warnings
CFLAGS  = -g -Wall

# source file (could actually list multiple .c files here)
SRCS = myprog.c

# define the executable file (assume source is same name):
TARGET = myprog

# this is generic:
all: $(TARGET)

$(TARGET): $(SRCS)
	$(CC) $(CFLAGS) -o $(TARGET) $(SRCS)

clean:
	$(RM) $(TARGET)

This Makefile can also be used for building an executable from more than one .c source file by just defining SRCS to list each c file. For example, if the executable is built from two source files myprog.c and util.c, define SRCS to be:

SRCS = myprog.c util.c

One thing to note about this Makefile is that it always recompiles the executable from all source files listed in SRCS if any one of them have been modified (because of this dependency: $(TARGET): $(SRCS)). For small programs, that may consist of just one or a few .c files, this may be fine. However, we can adjust the Makefile to have it only rebuild parts that need to be rebuilt when a source file is modified. We do this by adding rules to compile object files (.o) from each .c file, and then to build the executable file from linking the .o files (and any libraries) into the executable.

17.5.9. Advanced: A Generic Makefile for Multiple Files

We can rewrite the Makefile from Section 17.5.5 that compiled an executable from multiple source files as a more generic Makefile by making use of Makefile variables and pattern replacement rules.

Note that the rules for building the .o files from the .c in the orginal Makefile (in Section 17.5.5) are all similar except for the specific file names. It would be nice if we could simplify this with a more generic rule for building .o from .c files.

One way to do this is to use a pattern-replacement rule that is triggered for any .o file built from a .c file. Below is an example of such a rule. It uses makefile automatic variables $< for the name of the dependency of the rule (a .c source file) and $@ for the target file (the .o object file). See the gnu make manual for more information about automatic variables.

# This is a pattern replacement rule for building .o files from their
# associated .c files.  It uses automatic variables:
#   $<  the name of the first dependency of the rule (a .c file)
#   $@  the name of the target of the rule (a .o file)
%.o : %.c
        $(CC) $(CFLAGS) -c $< -o $@

This is a much shorter rule, and a generic rule, that will work for any C Makefile.

However, we can do even better than this! Because compiling a .o from a .c file is such a common build pattern, make has an implicit rule for building .o files from .c files. If we want to use this implicit rule, we don’t add any rule for building .o files from .c files to our makefile. The implicit rule is the following:

# make's implicit rule for building .o files from .c:
%.o : %.c
	$(CC) $(CPPFLAGS) $(CFLAGS) -c $< -o $@

The implicit rule uses some makefile variables that have default values, but that can also be redefined in the Makefile (and usually are):

  • CC: defines the compiler (our Makefile defines CC to be gcc)

  • CFLAGS: defines compiler flags (our Makefile defines these to be -g -Wall which includes debugging information and turns on all warnings).

  • CPPFLAGS: defines preprocessor flags (our Makefile doesn’t define any). The preprocessor is the first part of the compiler that runs. It expands anything starting with a #, like #include. Here is some more information about the the phases of compilation.

Because we have a rule for building $(TARGET) that has a dependency $(OBJS), make will look for a rule for building object files (.o) from source files (.c files), and it if doesn’t find one in the Makefile, it uses its implicit rule. If the implicit rule suffices, we don’t have to include any rule for building object files in our Makefile! Instead we just define the makefile variables it uses (or use its default definitions). If we don’t like the implicit rule, then we need to override it using our own rule.

The following is a version of the Makefile that uses the implicit rule for building .o files (and note that everything in the Makefile starting with OBJS = is generic!):

# Example Makefile using make's implicit .c:.o rule
#   make: build count executable program
#   make clean: clean-up all built files

# define the compiler
CC = gcc

# define compiler flags
CFLAGS  = -g -Wall

# define the executable file
TARGET = count

# define source files
SRCS = countwords.c counter.c scanner.c

# define the object files
# This uses Suffix Replacement within a macro:
#   $(name:str1=str2): for each word in 'name' replace 'str1' with 'str2'
# Here replace the suffix .c of all words in the macro SRCS with .o suffix
#
OBJS = $(SRCS:.c=.o)

# typing 'make' will invoke the first target entry in the file
all: $(TARGET)

# To create the executable file count we need the object files
# countwords.o, counter.o, and scanner.o:
#
$(TARGET): $(OBJS)
	$(CC) $(CFLAGS) -o $(TARGET) $(OBJS)

# using implicit rule to build .o files from .c!

# clean up built files
clean:
	$(RM) -f $(TARGET) *.o *~

This is a short and generic Makefile. However, there is one subtle problem with this that we don’t have with the longer less generic version: we have lost the dependencies of the header files on the object files, meaning that if the user updates a header file and types make the executable and .o files will not be rebuilt, but we want them to be. To fix this problem, we can add additional dependencies to the Makefile for TARGET and OBJS or for individual object files:

# define a variable to be the set of header files
HDRS = scanner.h counter.h

# add a dependency to this rule:
$(TARGET): $(OBJS)  $(HDRS)

# add a generic dependency on all $(OBJS)
# (note: no command part to this rule, it is just defining a dependency):
$(OBJS): $(HDRS) $(SRCS)

The resulting makefile looks like:

# Example Makefile with implicit .c:.o rule and header file dependencies
# make: build count executable program
# make clean: clean-up all built files

# define the compiler
CC = gcc

# define compiler flags
CFLAGS  = -g -Wall

# define source files
SRCS = countwords.c counter.c scanner.c

# define the object files
# This uses Suffix Replacement within a macro:
#   $(name:str1=str2): for each word in 'name' replace 'str1' with 'str2'
# Here replace the suffix .c of all words in the macro SRCS with .o suffix
#
OBJS = $(SRCS:.c=.o)

# define the header files
HDRS = scanner.h counter.h

# define the executable file
TARGET = count

# typing 'make' will invoke the first target entry in the file
all: $(TARGET)

# To create the executable file count we need the object files
# countwords.o, counter.o, and scanner.o:
#
$(TARGET): $(OBJS) $(HDRS)
	$(CC) $(CFLAGS) -o $(TARGET) $(OBJS)

$(OBJS): $(HDRS) $(SRCS)

# clean up built files
clean:
	$(RM) -f $(TARGET) *.o *~

With added dependencies of .h files added, changes to either a .c or a .h file will trigger rebuilding of .o and executable files. There is, however, one subtle inefficiency with this solution. The problem is with the generic dependency of objects on all sources and headers, which means that some .o files will be unnecessarily rebuilt when a specific .h or .c file is updated. For example, scanner.o will be rebuilt if counter.h is updated, but it does not need to be. The solution to this is to more explicitly enumerate the specific dependencies on each .o file in the Makefile, making the dependencies less generic and more specific for each .o file. For example, replace $(OBJS): $(HDRS) $(SRCS) in the above Makefile with:

count.o: count.c count.h
scanner.o: scanner.c scanner.h
countwords.o: countwords.c scanner.h counter.h

Adding these specific dependencies makes the Makefile less generic, but it avoids unnecessary compilation with the more generic $(OBJS): $(HDRS) $(SRCS). For small programs the more generic solution might be fine, but for larger programs carefully enumerating the dependencies for each .o file is important to avoid a lot of unnecessary re-compilation, and also to ensure that a .o file is rebuilt when a file it depends on is modified.

Because of this, there are ways to automatically generate a full set of dependencies for each .o file (and for each executable). Using automatic dependency generating tools is almost necessary for Makefiles for building programs from many source and header files in order to ensure correct compilation when a file is modified. One such tool is makedepend, which examines .c and .h files and adds dependencies to the end of a Makefile. Another is to use the gcc -MD (make dependencies) precompiler flag to generate dependency files (.d) and to include these files in the Makefile.

17.5.10. Advanced: A Generic Featureful Makefile

Our last makefile example is a generic general-purpose makefile that includes a more complete set of commonly used Makefile variables for compiling C programs, and it also demonstrates some more advanced syntax and features, specifically:

  • linking in libraries (-l)

  • defining link and include paths (-L and -I) for finding library and header files that are not in default locations.

  • auto generating file dependency rules (-MD) and including them in a makefile (include).

This is also a good example Makefile to copy and edit for your own use

In general, C programs have many libraries implicitly linked into them, but sometimes other libraries need to be explicitly linked in using the -l compiler flag. The gcc compiler looks in default locations for library code, but for libraries that are not in those locations, a link path needs to be explicitly included in the gcc command line. These are added to the makefile using -L. Similarly, header files also may be located in non-standard locations, and then the Makefile needs to specify an include file path definition to the location containing them. Include paths are given using -I.

Finally, for a program with multiple .c and .h files, it is useful to automatically generate dependencies. In this example we use the gcc -MD (make dependencies) precompiler flag (added to the CPPFLAGS definition). It creates dependency files (.d) associated with each .o file during the preprocessing phase of compilation (the first phase). These dependency files are then included in the Makefile (include). There is an implicit dependency of .d files on .c files, so if a .c file is modified both its .o and .d are regenerated. Also, the -MP flag is typically used with the -MD flag, which suppresses errors if header files are renamed.

Here is the resulting Makefile, which is a good generic makefile to copy and modify for your own use:

# Example Makefile to copy and edit for your own use
#   make: build .d, .o, and executable target
#   make clean: remove built files

# define compiler
CC = gcc

# define compiler flags
CFLAGS = -Wall -g

# define include paths:
#  define any non-default paths to header files (.h) used by this program
#  (default location, /usr/include/, is searched by gcc last)
INCLUDES =  -I../includes -I.

# define preprocessor flags
# -MD and -MP flags create dependency files (.d) for all SRCS
CPPFLAGS = -MD -MP $(INCLUDES)

# library paths
#  define any non-default paths to where libraries used by the program may be
#  (default location, /usr/lib/, is searched by gcc last)
LFLAGS = -L../lib -L/home/newhall/lib

# define libraries to link into executable
#    -lm: std math library in /usr/lib (named libm.so or libm.a)
#    -lmylib: is a library not in standard location, so -L paths
#             are used to find it (named libmylib.so or libmylib.a)
LIBS = -lm -lmylib

# set of source files
#  (\ is the line continuation character)
SRCS =  emitter.c init.c lexer.c  symbol.c parser.c ast.c  \
        error.c symbol_table.c codegen.c emitcode.c main.c

# set of object files
#  obtained from SRCS using suffix replacement rule
OBJS = $(SRCS:.c=.o)

# executable file name
TARGET = mycc

# .PHONY tells make that these targets are not file names but
#   the target labels for a set of commands to run when `make [target label]`
#   is invoked (e.g., `make clean` executes a `rm` command)
#
.PHONY: clean

# the rest of this is generic and should not need to be edited:
all:    $(TARGET)

$(TARGET):  $(OBJS)
	$(CC) $(CPPFLAGS) $(CFLAGS) $(INCLUDES) -o $(TARGET) $(OBJS) $(LFLAGS) $(LIBS)

# just use make's built-in rule (.c.o:) to build .o from .c files!

clean:
        $(RM) *.o *~ $(TARGET) *.d

# include the dependency files here
#   this uses a suffix replacement within a macro $(name:string1=string2):
#     (for each word in 'name' replace 'string1' with 'string2')
#     here we replace the suffix .c with .d for every name in SRCS
#
#   the '-' before 'include' suppresses error messages when a .d file
#     doesn't already exist
-include $(SRCS:%.c=%.d)

The INCLUDES variable defines paths to search to find .h files that are needed to compile .c files to .o for any .h. The path only needs to be specified if .h files are not in the current working directory or in locations that gcc checks by default.

During the linking phase of compilation, the compiler links the math library -lm and a local library -lmylib defined by the LIBS variable that it finds given the library path definitions in the LFLAGS variable specifying paths to search. The path only needs to be specified in cases where the library file is not in locations that gcc automatically searches by default.

Running make using this makefile will first generate .d files for each .c file because we define CPPFLAGS to have -MD, which means make dependency files. If you run ls you will see these .d files:

$ make
$ ls
Makefile ast.c ast.d ast.o codegen.c codegen.d codegen.o ...

The dependency files are included in the Makefile in the last line:

-include $(SRCS:%.c=%d)

This uses a suffix replacement rule (SRC:%.c=%d) to specify including every .d file, and the - before include suppresses error messages if the .d file doesn’t already exist. For more details, see automatically generate makefile dependencies from microHOWTO.

This Makefile also uses make’s built-in rule to build .o files from .c files:

.c.o:
    $(CC) $(CPPFLAGS) $(CFLAGS) $(INCLUDES) -c $<  -o $@

Because we set CPPFLAGS to include the INCLUDES as well as the command options to create the .d dependency files, we can use the built-in rule.

If we didn’t add INCLUDES to CPPFLAGS, then we could override the built-in rule in the Makefile with our own (sometimes it is necessary to override a built-in command like this, but this is not an example where it is):

...
INCLUDES = -I../includes/ -I.
CPPFLAGS = -MD -MP
...
# override built-in command with our version:
.c.o:
        $(CC) $(CPPFLAGS) $(CFLAGS) $(INCLUDES) -c $<  -o $@

17.5.11. Advanced: Makefile Generators

For very large programs, and particularly for software that may be installed on different systems, using a program that generates a Makefile for a particular system makes managing different build targets considerably easier. GNU Automake (part of GNU Autotools library) and CMake are two examples of makefile generating programs. Both take user-specifications in a config file to find libraries and create dependencies on specific systems and auto generate Makefiles for that system. We don’t discuss either here, but our references section has links to more information about both.

17.5.12. References

Here are a few references for using make and writing makefiles: