Fuzzing Qt with libFuzzer
Disclaimer: I had this blog post in the drawer for several weeks now; I just held back with publishing it to give the Qt security team time to fix the reported issues (see below under 4.).
Inspired by the great talk from Hanno Böck about American Fuzzy Lop and Address Sanitizer at QtCon, having a try on fuzzing Qt itself seemed like an interesting thing to do. For fuzzing different parts of a library like Qt, libfuzzer is a good choice: In contrast to e.g. American Fuzzy Lop, libfuzzer runs the test cases in the same process and does not need to spawn a new process for each test, which makes it very fast.
These are the steps to set up libFuzzer and test it on Qt:
1. Compile Qt with address sanitizer and code coverage information:
To get proper output when something is wrong (e.g. "heap user after free"), Qt needs to be built with address sanitizer; in addition, libFuzzer collects information about which code paths it has visited. To enable those two settings with clang on Linux, the following patch has been applied:
diff --git a/mkspecs/linux-clang/qmake.conf b/mkspecs/linux-clang/qmake.conf index 77d913f..03d4c47 100644 --- a/mkspecs/linux-clang/qmake.conf +++ b/mkspecs/linux-clang/qmake.conf @@ -11,6 +11,8 @@ include(../common/linux.conf) include(../common/gcc-base-unix.conf) include(../common/clang.conf) -QMAKE_LFLAGS += -ccc-gcc-name g++ +QMAKE_LFLAGS += -ccc-gcc-name g++ -fsanitize-coverage=edge -fsanitize=address +QMAKE_CFLAGS += -fsanitize-coverage=edge -fsanitize=address +QMAKE_CXXFLAGS += -fsanitize-coverage=edge -fsanitize=address load(qt_config)
Even without using a fuzzer, compiling Qt and an application with address sanitizer might highlight problems within the code.
Apparently the settings above are also available for gcc; also, it might be possible to specify those changes on the command line instead of patching the mkspec file.
2. build libfuzzer:
Libfuzzer is part of llvm, and has a tutorial on how to build it. The resulting libFuzzer.a needs to be linked to the executables, as explained in the steps below.
3. write test cases:
For the fuzzing test cases, there is one function to be implemented, which will be called automatically by the fuzzer. Here is an example of a simple Qt test case:
#include <QtCore> #include <QtSvg> extern "C" int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) { QByteArray byteArrayData(reinterpret_cast<const char *>(Data), Size); QSvgRenderer renderer(byteArrayData); return 0; }
In particular, the test programs do not need a main() function, as it is already contained in libFuzzer.a. Here is the .pro file for the test case above:
TEMPLATE = app TARGET = QSvgRenderer INCLUDEPATH += . QT += svg # Input SOURCES += qsvgrenderer.cpp LIBS += -L/home/peter/dev/fuzzers -lFuzzer
The full set of Qt test cases written so far can be found on github:
This repository contains tests for QImage, QJsonDocument, QtNetwork and QXmlStreamReader, among others.
4. run and report results:
The test cases can be run using all available cores via e.g. "./QSvgRenderer -jobs=4 -workers=4 testcases". Also, the github repository above contains an initial set of test cases the fuzzer can start with. For instance when fuzzing SVG images, it makes sense to start with a valid SVG file as initial input.
The tests from the repository produced some issues in Qt, which have been reported to the Qt security mailing list or the Qt bugtracker (for stack traces and information on how to reproduce see the links):
QtDbus heap use-after-free (reported to the security mailing list on 16th of October 2016)
QJsonDocument heap buffer overflow (reported to the security mailing list on 19th of October 2016)
memory leak when creating an invalid SVG file reported to the Qt bug tracker on (23rd of October 2016)
Next steps:
A good way to improve results seems to look into user-supplied mutators to have more control over which input is fed into the test cases. E.g. flipping bits seems like a good default way for binary data such as images, but when it comes to JSON or XML, there might be smarter ways to produce faulty input.
