Secure Coding Practices, Detecting Bugs

You can access the slides 🖼️ for this lecture.

Here we review methods to try to minimise the number of bugs we introduce when writing systems software.

Context

We have seen that memory safety and undefined behaviour issues are common in systems software, and that they lead to security vulnerabilities that can be exploited by attackers to compromise the confidentiality, integrity, and availability of systems software. How can we address that problem? In practice, we can do three things:

1. When we develop we need to adhere to good coding practices to minimise the chances of introducing such bugs.
2. We have techniques that can help analyse our code during development and detect some bugs.
3. We also have techniques that can help protect programs in production, making exploitation more difficult and limiting the damage from successful exploits.

Here we discuss 1 and 2, and the next chapter will cover 3.

Good Coding Practices

As we have seen, the memory errors and other sources of undefined behaviour that lead to security vulnerabilities come from programming mistakes. When developing systems software, how can we avoid introducing these programming mistakes as much as possible? There are two main aspects to this problem:

• First, how to avoid introducing these programming mistakes when writing code?
• Second, how to detect these programming mistakes in existing code?

We cover both aspects next, starting with secure coding practices.

Array/Buffer/Integer Overflows

To prevent array or buffer overflows, you need to know their sizes: you must be aware of the size of an array to know when to stop iterating over it, and of the size of a destination buffer to know how many bytes you can write in there. In C, recall that arrays and buffers do not embed their sizes, so make sure to keep track of the size of each array/buffer you use.

When manipulating integers, make sure to be aware of the size reserved by the compiler to hold them in memory according to the architecture you are compiling for, in order to avoid overflows. You can use sizeof to determine these sizes. An unsigned integer will never overflow but rather wrap around; overflowing a signed integer leads to undefined behaviour and must be avoided. The compiler has some built-in functions that can tell you if an integer operation overflows. These operations are available for integer addition, subtraction, and multiplication.

C Standard Library Functions

Unsafe Functions to Avoid. Here are a few functions of the libc whose use should be avoided as much as possible:

The table explains the reason why each function is unsafe, as well as safe alternatives. Several of them, such as strcpy, have no bounds checking, hence no way to prevent overflows. You should use the safe versions of these functions as much as possible, which all have a way to indicate the size of the receiving buffer to avoid overflows. To move memory you should not rely on bcopy but rather use memcpy (with care, i.e. you have to handle bounds checking) if the source and target areas do not overlap, and memmove if they do. Finally, be careful with strlen: it can return numbers larger than the size of a string if that string is not properly terminated.

These are not the only functions to avoid; please see here for more.

String Manipulation Functions. Once again, regarding string manipulation functions, careful developers should use the n versions that require indicating a maximum number of characters to process: strncpy rather than strcpy, snprintf instead of sprintf, etc. Even with the versions with n, there are some particularities to keep in mind, for example strncpy will not add the termination character \0 at the end of the target buffer.

Consider this code in which we wish to replace the content of string2, that is composed of 32 x’s, with "hello, world".

char string1[] = "hello, world";
char string2[32] = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx";

strncpy(string2, string1, strlen(string1));
printf("%s\n", string2); // prints "hello, worldxxxxxxxxxxxxxxxxxxx"

Because strncpy does not add the termination character, we end up with a mix of both strings which is probably not what the programmer intended.

Dynamic Memory Allocation

When using dynamic memory allocation, make sure to always check malloc’s return value, for reasons we have previously discussed. Also remember that, after free is called upon a pointer, that pointer is invalid and should not be reused in any way. It should obviously not be dereferenced, but its value should not be used for any other purpose.

As we have seen previously, realloc returns NULL upon failure but does not free the old pointer, so the following code is in effect a memory leak:

ptr = realloc(ptr, new_size);

malloc does not zero out memory returned to allocation requests, so if you only partially initialise a data structure located in a dynamically allocated buffer, and you pass that data structure to a context that you do not trust (for example by sending it through the network), you may be leaking memory content to that untrusted party. So for buffers sent to untrusted contexts, it is better to use calloc, which will zero out the memory it allocates, at the cost of a performance slowdown.

Secure Coding: Further Readings

What we have seen is just a few examples of secure coding practices, and we do not have time to cover them all exhaustively. You can see on the slides a list of good resources; make sure to check them out if you want to learn more:

• SEI CERT C Coding Standard: https://wiki.sei.cmu.edu/confluence/display/c
• Robert C. Seacord, Secure coding in C and C++ (book)
• ISO/IEC TS 17961 (C Secure Coding Rules): https://www.open-std.org/jtc1/sc22/wg14/www/docs/n1624.pdf
• NASA JPL C Coding Standard: https://yurichev.com/mirrors/C/JPL_Coding_Standard_C.pdf
• Fedora’s Defensive Coding Guide: https://docs.fedoraproject.org/en-US/defensive-coding/

Detecting Coding Mistakes

Let’s now investigate analysis tools that can help detect programming mistakes and vulnerabilities in existing code. Here we will cover techniques that are slow to execute, or that make the application slow. As a result, they cannot run in production, and are rather used during development, often integrated with applications’ CI/CD pipelines.

These techniques fall within two main categories:

• Static analysis approaches
• Dynamic analysis approaches

Static Analysis

Static analysis tools scan the source code of a program for possible bugs without actually running the program. An important benefit of this approach is that it has good coverage: it analyses the entire program’s code, which lends itself well to automation. In terms of downsides, static analysis generally suffers from false positives: this means it may identify issues in the code that actually do not represent programming mistakes or security vulnerabilities. Because it does not run the program, static analysis also operates with limited context: for example, much of the memory’s content is not determined until runtime. Finally, some static analysis techniques are quite slow, and do not scale well to the large code bases of certain systems software.

Compiler Warnings and Errors. A first form of static analysis is compiler warnings and errors. A careful programmer should enable high degrees of compiler warnings. The compiler flags that can be used are, in increasing order of strictness:

• -Wall to get additional warnings.
• -Wextra to get even more warnings.
• -pedantic to add even more warnings.

High degrees of strictness may yield a high number of false positives. The -Werror flag will transform warnings into errors: the first warning encountered will stop the compilation process, forcing the programmer to fix it to be able to build the program.

To understand what types of warnings are added by each option, see GCC’s relevant documentation.

Code Static Analysis Tools. There are many advanced static analysis tools; a few examples are:

• The Clang static analyser
• Lint
• Coverity
• cppcheck

Next, we present an example of using the Clang static analyser. Consider the faulty program below; it contains three bugs:

int c;

int main() {

    int a = INT_MAX;
    int b = 1;
    c = a + b; // Integer overflow!

    char buffer[8];
    char str[] = "this string is too long";
    strcpy(buffer, str); // Buffer overflow!


    int *ptr = (int *)malloc(sizeof(int));
    *ptr = 42;
    free(ptr);
    *ptr = 99; // Use-after-free!

    return 0;
}

The first bug is an integer overflow: c’s value is set to the largest integer that can be stored in an int, INT_MAX, then it is incremented, which triggers the overflow. The second bug is a buffer overflow: we copy into buffer (whose size is 8 bytes) a string str that is larger than 8 bytes. And the last bug is a use-after-free, where we dereference the pointer ptr after freeing the memory it points to.

Notice that with the default level of warnings, this program compiles fine, and also it runs without any visible error:

$ gcc faulty.c -o faulty
$ ./faulty

Let’s see if the bugs it contains can be detected with the Clang static analyser. To run the analyser, invoke it as follows:

$ clang --analyze faulty.c
faulty.c:22:10: warning: Use of memory after it is freed [unix.Malloc]
    *ptr = 99; // Use-after-free!
    ~~~~ ^
1 warning generated.

The analyser detects the use-after-free, which is good. However, it does not identify the integer and buffer overflows. For that we need to rely on the second main class of analysis tools: dynamic analysis.

Dynamic Analysis

Dynamic analysis tries to detect errors while running the program. By doing so, it gets access to more information than static analysis, that is runtime information (for example the content of memory or the value of program input). Dynamic analysis is also useful when the sources of the program we wish to analyse are not available (black-box testing).

A very popular type of dynamic analysis is achieved through compiler-based instrumentation: at build time the compiler inserts additional instructions in the program to detect bugs later at runtime. The code sanitisers are a series of tools enabling such dynamic analysis. The most widespread is address sanitiser (ASan). ASan will detect a wide range of memory errors that would not be caught at compile-time or at runtime without the instrumentation. We also have the undefined behaviour sanitiser (UBSan), that detects things like integer overflows, invalid casts, and so on, or the thread sanitiser, that will detect concurrency issues such as race conditions.

More information on code sanitisers is available here.

Sanitisers: ASan and UBSan. We can illustrate the use of ASan and UBSan by running these analysis tools on our faulty program. To instrument the program with ASan, compile it as follows:

$ clang -fsanitize=address faulty.c -o faulty

Then simply launch the program normally:

$ ./faulty
=================================================================
==21543==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ffcc881f268
...

As one can see, ASan can catch the buffer overflow. Once that buffer overflow is fixed, we can recompile, still with ASan enabled, and re-run the analysis:

clang -fsanitize=address faulty.c -o faulty
$ ./faulty
./faulty                                   
=================================================================
==22504==ERROR: AddressSanitizer: heap-use-after-free on address 0x602000000010 # ...

This time we can see that the use-after-free is detected. We can also enable UBSan:

$ clang -fsanitize=undefined faulty.c -o faulty
$ ./faulty
faulty.c:12:11: runtime error: signed integer overflow:
    2147483647 + 1 cannot be represented in type 'int'

UBSan successfully detects the integer overflow.

Valgrind. There are other dynamic analysis tools beyond the sanitisers, although many of such tools have been rendered more or less obsolete by them. We have seen Valgrind previously. In addition to reporting memory leaks, it can also detect certain memory errors. Given that sanitisers also detect memory leaks, that makes Valgrind quite redundant. However, one benefit over sanitisers is that Valgrind does not require recompiling the program to insert instrumentation. Hence, Valgrind is still useful in contexts where we have access only to the application’s binary and not its sources (e.g., black-box testing of proprietary software).

Fuzz-Testing. A highly popular modern dynamic analysis technique is fuzzing (sometimes referred to as fuzz-testing). It consists in blasting a trust boundary with malformed inputs (e.g., pseudo-random data) with the hope of triggering bugs. Examples of trust boundaries that are good candidates for fuzzing include command-line arguments, input files, network packets, and so on. Fuzzing is widespread today, and this technique helps uncover a very large number of bugs in many projects.

Let’s see an example of fuzzing with the tool American Fuzzy Lop (AFL). We will fuzz the following vulnerable program:

int main(int argc, char *argv[]) {
    char name[32];  // Vulnerable buffer (too small for unchecked input)

    if (argc < 2) {
        printf("Usage: %s <input file>\n", argv[0]);
        return 1;
    }

    FILE *f = fopen(argv[1], "r");
    if (!f) {
        printf("Error, can't open %s\n", argv[1]);
        return 1;
    }

    fread(name, 1, 512, f);  // Reads up to 512 bytes into a 32-byte buffer!
    fclose(f);

    printf("hello %s\n", name);
    return 0;
}

This program opens a file and reads its content into a buffer. It reads 512 bytes; however, the destination buffer is only 32 bytes long, so there is a possibility of overflow here. The name of the file to read comes from the command line. We are going to fuzz this program by invoking it repeatedly and injecting files with random data and variable sizes, which should help trigger the bug.

To install AFL on an Ubuntu/Debian distribution:

$ sudo apt install afl # or afl++ on very recent ubuntu/debian distributions

We first need to compile and instrument our target program for fuzzing:

$ afl-clang fuzzme.c -o fuzzme

In addition to inserting the necessary code for fuzzing, the instrumentation will also enable sanitisers to maximise the number of bugs discovered. Before being able to fuzz, we also need to create a seed input to help kickstart the fuzzing process:

$ mkdir input
$ echo "testname" > input/seed

Finally, we can start the fuzzing process as follows:

$ AFL_SKIP_CPUFREQ=1 afl-fuzz -i input -o output -- ./fuzzme @@

AFL’s window will report, in real time, statistics about the fuzzing process: runtime, number of program invocations, fuzzing strategy used, and so on. Pay attention in particular to the total crashes : field: there should be at least one crash discovered very quickly after the start of the fuzzing process. Crashes represent potential bugs (e.g., ASan crash) found by the fuzzer. By default, AFL will fuzz indefinitely: to terminate the fuzzing process, hit ctrl+c on the keyboard.

To reproduce manually a particular crash uncovered by the fuzzer, compile the program normally with ASan enabled, and inject the payload corresponding to that crash:

$ clang -g -fsanitize=address fuzzme.c -o fuzzme
$ ./fuzzme output/crashes/id:000000,sig:11,src:000000,op:havoc,rep:128
...
==161882==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7b2b61bf0040 at pc 0x559b133ffbac bp 0x7ffc92be10f0 sp 0x7ffc92be08b0
WRITE of size 36 at 0x7b2b61bf0040 thread T0
    #0 0x559b133ffbab in fread.part.0 asan_interceptors.cpp.o
    #1 0x559b13531ff0 in main /home/pierre/Desktop/comp60261/slides/10-secure-coding-practices-detecting-bugs/src/fuzzme.c:20:5
...

Note that the name of the payload file may be different on your computer. As one can see, the payload successfully triggered the buffer overflow present in our buggy program.

There is a lot more to say about fuzzing. It’s a field that has seen a lot of recent developments. A few relevant further readings:

• Fuzzing: Brute Force Vulnerability Discovery
• The Fuzzing Book
• Fuzzing 101

Other Static and Dynamic Analysis Approaches. There are a few other static and dynamic analysis techniques that can be used to detect programming mistakes, bugs, and vulnerabilities. You are probably familiar with unit testing and manual code reviews, as well as with tools to check that code follows a certain style (linters/style checkers). There are other advanced techniques that we won’t cover here, but have shown good results:

• Taint analysis
• Symbolic execution
• Abstract interpretation
• Formal verification