Detecting Bugs

You can access the slides 🖼️ for this lecture. All the code samples given here can be found online, alongside instructions on how to bring up the proper environment to build and execute them here.

We previously saw how to adopt some good practices when writing code to avoid introducing bugs that could translate in security issues. Here we are going to cover a complementary approach, which is the use of automated tools to detect programming mistakes and bugs into existing code bases.

Detecting Coding Mistakes

Here we cover techniques that are slow to execute, or that make the application slow. As a result they cannot run in production, and are rather used during development. These techniques fall within 2 main categories:

Static analysis, which consists in scanning program's source code or binary for bugs without executing it.
Dynamic analysis, which consists in checking for the presence of bugs by running the program.

Static Analysis

Static analysis tools scan the source code of the program for possible bugs without actually running the program. The benefit of this approach is that it has good coverage, it goes over the entirety of the program's code, which lends itself well to automation. In terms of downsides, static analysis generally suffers from false positives: it means it may identify issues in the code that actually do not represent programming mistakes or security vulnerabilities. Because it does not run the program, the efficiency of static analysis suffers from a limited amount of context, for example most of the memory content is not determined until runtime. Finally, some static analysis techniques are quite slow and do not scale well to the large code bases of certain systems software.

Many static analysis approaches are implemented at the level of the compiler. A first thing you should do is enable high degrees of compiler warnings. You can use, by increasing order of pickiness, -Wall to get additional warnings, -Wextra to get even more warnings, and -pedantic to add even more warnings. See here for details about what warnings are added by each option.

A few example of static analysis tools helping to detect coding mistakes in C programs include:

The Clang static analyser
Linthttp://squoze.net/UNIX/v7/files/doc/15_lint.pdf
Coverity
Cppcheck
Among others

Let's go over an example with the Clang static analyser.

The Clang Static Analyser

Consider the following code:

// includes omitted

int c;

int main() {

    int a = INT_MAX;
    int b = 1;
    c = a + b; // Integer overflow!

    char buffer[8];
    char str[] = "this string is too long";
    strcpy(buffer, str); // Buffer overflow!


    int *ptr = (int *)malloc(sizeof(int));
    *ptr = 42;
    free(ptr);
    *ptr = 99; // Use-after-free!

    return 0;
}

This program is faulty and contains 3 bugs:

The first bug is an integer overflow, we add in c 1 to the largest integer that can be stored on an int INT_MAX.
The second bug is a buffer overflow, we copy in buffer, which size is 8 bytes, a string that is larger than 8 bytes.
And the last bug is a use after free, where we dereference the pointer ptr after having freed the buffer it points to.

Notice that with the default level of warnings, this program compiles fine, and also it runs without any visible error. Can static analysis help detect these issues? If we run the Clang static analyser over our program, we get the following output:

$ clang --analyze faulty.c
faulty.c:22:10: warning: Use of memory after it is freed [unix.Malloc]
    *ptr = 99; // Use-after-free!
    ~~~~ ^
1 warning generated.

The tool is able to detect the use after free bug, however it does not detect the two other bugs. For that we need to use dynamic analysis.

Dynamic Analysis

Dynamic analysis tries to detect errors while running the program. Doing so it gets access to more information than static analysis, that is runtime information. It's also useful when the sources of the program we wish to analyse are not available.

A very popular type of dynamic analysis is achieved through compiler based instrumentation. These are called sanitisers. The most widespread is address (ASan), that will detect a wide range of memory errors that would not be caught at compile time or at runtime without the instrumentation. You also have the undefined behaviour sanitiser (UBSan) that detects things like integer overflows, invalid casts, and so on. Check out this link for more information about sanitisers.

Using Sanitisers

We can enable address sanitiser instrumentation at compile time on our faulty program as follows:

$ clang -fsanitize=address faulty.c -o faulty

Next, if we launch the program, we can see that ASan catches the buffer overflow:

$ ./faulty
=================================================================
==21543==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ffcc881f268 # ...

Once we have fixed that particular overflow we can recompile the program, still with address sanitiser, and launch it again. This time we can see that it detects the use after free:

clang -fsanitize=address faulty.c -o faulty
$ ./faulty
./faulty                                   
=================================================================
==22504==ERROR: AddressSanitizer: heap-use-after-free on address 0x602000000010 # ...

And finally, when we enable undefined behaviour sanitiser, and launch the program, the integer overflow is detected:

$ clang -fsanitize=undefined faulty.c -o faulty
$ ./faulty
faulty.c:12:11: runtime error: signed integer overflow:
    2147483647 + 1 cannot be represented in type 'int'

Valgrind

There are other dynamic analysis tools available. Most of what came before the sanitisers has been rendered more or less obsolete by them. We have seen Valgrind previously, in addition to reporting about memory leaks, it can also detect certain memory errors. Given that sanitisers also detect memory leaks, that makes Valgrind quite redundant. However, note that with Valgrind there is no need to recompile the program to insert instrumentation, as one do with the sanitisers. So Valgrind is still useful in context where we have only access to the application's binary and not its sources.

Fuzz Testing AKA Fuzzing

Fuzzing consists in blasting a trust boundary with malformed inputs with the hope to trigger bugs. Examples of trust boundaries that are good candidates for fuzzing include the command line arguments, input files, network packets, and so on. Fuzzing is highly popular these days, and has help uncover a very large number of bugs in many projects.

Let's see an example of fuzzing with the widely popular toot American Fuzzy Lop (AFL).

Consider this vulnerable program:

int main(int argc, char *argv[]) {
    char name[32];  // Vulnerable buffer (too small for unchecked input)

    if (argc < 2) {
        printf("Usage: %s <input file>\n", argv[0]);
        return 1;
    }

    FILE *f = fopen(argv[1], "r");
    if (!f) {
        printf("Error, can't open %s\n", argv[1]);
        return 1;
    }

    fread(name, 1, 512, f);  // Reads up to 512 bytes into a 32-byte buffer!
    fclose(f);

    printf("hello %s\n", name);
    return 0;
}

This program opens a file and reads its content into a buffer. It reads 512 bytes, however the destination buffer is only 32 bytes long, so there is a possibility of overflow here. The name of the file to read comes from the command line.

By fuzzing this program with AFL, we are going to inject many different types of files as its first command line argument, and hope that some will trigger the bug.

To install AFL on a Debian/Ubuntu system, install this package:

$ sudo apt install afl # or afl++ on very recent ubuntu/debian distributions

Next let's compile our vulnerable program and instrument it for fuzzing with AFL. That instrumentation will include enabling ASan to detect a maximum amount of bugs, and code coverage statistics to measure the progress of the fuzzing process. This is done with a compiler wrapper named afl-clang:

$ afl-clang fuzzme.c -o fuzzme

Before we start to fuzz we need to create some seed input file to kick-start the fuzzing process:

$ mkdir input
$ echo "testname" > input/seed

We can now start the fuzzing process:

$ AFL_SKIP_CPUFREQ=1 afl-fuzz -i input -o output -- ./fuzzme @@

After a few seconds, stop the fuzzing with ctrl+c. If AFL found any bugs, which should be the case for our example, the output (here, files) that generated the crashes will be saved in output/default/crashes. You can reproduce the crashes by compiling the target program normally, and injecting the output in question:

$ clang -g -fsanitize=address fuzzme.c -o fuzzme
$ ./fuzzme output/default/crashes/id:000000,sig:11,src:000000,op:havoc,rep:128

You should get a nice ASan crash report pointing out the overflow in the program.

Fuzzing is a vast field, here are a few pointers if you want to dig deeper:

Sutton et al., Fuzzing: Brute Force Vulnerability Discovery
The Fuzzing Book
Fuzzing 101: https://github.com/antonio-morales/Fuzzing101

Other Static and Dynamic Analysis Approaches

There are other static and dynamic analyses techniques that are worth mentioning. You are probably already familiar with unit testing and manual code reviews, as well as with tools to check that your code follows a certain style that maximises clarity and reduces the chances of introducing bugs.

There are other advanced/researchy techniques to find bugs or prove the correctness of software, such as taint analysis, symbolic execution, abstract interpretation, formal verification, or model checking.

To conclude, we covered the use of various automated tools to try to detect bugs in existing code. They belong in two main categories, static and dynamic analysis approaches. Unfortunately, even if you combine these with the secure coding practices we saw previously, none of these approaches will allow you to get rid of 100% of the bugs and vulnerabilities. We also need defences executing at runtime in production, to make exploits harder and limit their damage.

Keyboard shortcuts

COMP26020 Part 1: C Programming