Tutorial

Fuzzinator is a framework helping you to deal with the common fuzzing tasks, like running fuzz jobs, updating the targets, and reducing the inputs that induced failures. The figure below shows a high-level overview of the components of the framework.

_images/architecture.png

The red line represents an API boundary. Everything below it is part of the core infrastructure, while boxes above it are user-defined. However, you don’t (necessarily) need to write a single line of code to describe your needs, since Fuzzinator can be configured through configuration ini files and it comes with several built-in building blocks that cover the most common scenarios and can be used out of the box.

In the next paragraphs, we will use the JerryScript project as our running example and we will incrementally build a configuration file to setup a fuzzing infrastructure for it. We will start from an absolute minimum configuration that will be extended step-by-step.

Although, the examples cover only a small subset of the provided building blocks, you can find the full list in the API Reference (under sub-packages with descriptive names). If none of them fits your needs, then you can still write your own snippet … or submit a feature request ;-)

Minimum Configuration

Let’s start with the minimum configuration example that defines one SUT with fuzzinator.call.StdinSubprocessCall, expecting input from stdin and one test generator with fuzzinator.fuzzer.RandomContent that simply produces random strings.

# Sections starting with 'sut.' prefix define how target applications (a.k.a.,
# software-under-test or SUT) will be handled. The string after 'sut.' will be
# used as the identifier of the target. In this example, we deal with
# JerryScript.
[sut.jerry]
# StdinSubprocessCall will execute the target and return an issue dictionary if
# the target exits with a non-zero code.
call=fuzzinator.call.StdinSubprocessCall

# Define parameters expected by StdinSubprocessCall.
[sut.jerry.call]
# 'command' defines how SUT has to be executed.
command=./build/bin/jerry -
# Directory where 'command' has to be run.
cwd=</path/to/jerryscript/root/directory>

# Sections starting with 'fuzz.' prefix bind SUTs and test case generators.
[fuzz.jerry-with-random]
# Specify the SUT by referencing the appropriate config section.
sut=jerry
# Specify the fuzzer by referring a Python callable.
fuzzer=fuzzinator.fuzzer.RandomContent

Fine Tuning SUT Calls

Now, if you would like to fine-tune error detection to do more than simply checking for a non-zero exit-code, then you can use two built-in solutions (or, again, you can implement your own version):

We can extend the original example as follows:

[sut.jerry]
# ... define filters ...
# Properties named as 'call.decorate(N)' are Python decorators that can access
# the input & output of the wrapped methods (in this case, of
# StdinSubprocessCall) and can modify them. Here, they are used to filter the
# output issues. If decorators expect parameters, then they have to be defined
# in parameter sections named as 'sut.<SUT_NAME>.call.decorate(N)'.
call.decorate(0)=fuzzinator.call.ExitCodeFilter
call.decorate(1)=fuzzinator.call.RegexFilter

# Parameter section for ExitCodeFilter.
[sut.jerry.call.decorate(0)]
exit_codes=[132, 129]

# Parameter section for RegexFilter.
[sut.jerry.call.decorate(1)]
stderr=["(?P<msg>Assertion '.*' failed )at (?P<file>[^(]+)[(](?P<path>[^)]+)[)]:(?P<line>[0-9]+)",
        "(?P<msg>Unreachable control path )at (?P<file>[^(]+)[(](?P<path>[^)]+)[)]:(?P<line>[0-9]+)"]

However, issues not only can be filtered but also extended with arbitrary information that helps describing the circumstances of the failure. This extension can also happen with the above shown decorator approach. The next example shows how platform, git version, and ID information can be added using:

[sut.jerry]
# .. extend issue with platform information ..
call.decorate(2)=fuzzinator.call.PlatformInfoDecorator
# .. extend issue with user-defined information ..
call.decorate(3)=fuzzinator.call.SubprocessPropertyDecorator
# .. add an id to the issue ..
call.decorate(4)=fuzzinator.call.UniqueIdDecorator

# *No* parameter section for the 2nd decorator as it needs none.

# Parameter section for the 3rd decorator.
[sut.jerry.call.decorate(3)]
# .. extend issue dictionary with a version field ..
property=version
# .. the value of version field is filled with the output of the next command ..
command=git rev-parse --short HEAD
# .. directory where 'command' has to be run (no need to copy the value of 'cwd'
# from the 'sut.jerry.call' section verbatim, extended interpolation syntax can
# help to reuse options) ..
cwd=${sut.jerry.call:cwd}

# Parameter section for the 4th decorator.
[sut.jerry.call.decorate(4)]
# .. compose the new id field from the msg and path fields previously found by
# RegexFilter ..
properties=["msg", "path"]

Updating SUTs and Reducing Tests

Similarly to the above, we can have control over SUT update and test reduce jobs as well. The following final example uses built-in building blocks again:

[sut.jerry]
# ... define update ...
update_condition=fuzzinator.update.TimestampUpdateCondition
update=fuzzinator.update.SubprocessUpdate
# ... define reduction ...
reduce=fuzzinator.reduce.Picire

# Parameter section for fuzzinator.update.TimestampUpdateCondition.
[sut.jerry.update_condition]
# Update SUT in every 12 hours.
age=12:00:00
path=${sut.jerry.call:cwd}/build/bin/jerry

# Parameter section for fuzzinator.update.SubprocessUpdate.
[sut.jerry.update]
# Script to execute to update.
command=git pull origin master &&
        ./tools/build.py --debug --clean
# Directory where 'command' has to be run.
cwd=${sut.jerry.call:cwd}

Etc…

There is more, e.g.:

  • SUTs can take their input from files instead of stdin.

  • Reducers are highly parametrizable.

  • Test reduce jobs can deviate from fuzz jobs in the way their SUT is called.

  • Fuzzers can be decorated the same way as SUT calls.

  • Etc…

More complex configuration files are available in the examples/configs directory of the project (e.g., for WebKit, too).