SUT Calls: package fuzzinator.call

class fuzzinator.call.AdaptiveTimeoutDecorator(**kwargs)

Decorator for SUT calls used in validate or reduce jobs. It helps dynamically optimizing the timeout of SUT calls. It works with SUTs that receive and respect a timeout kwarg both in their __init__ and __call__ methods. Further requirement is that the issue dict provided by the decorated SUT call must contain time and id fields. Having these requirements fulfilled, the decorator adapts the timeout of the next call by taking the geometric mean of the previous two execution times that reproduced the expected issue.

The decorator takes no parameter.

Example configuration snippet:

[sut.foo]
call=fuzzinator.call.SubprocessCall
reduce_call=${call}
reduce_call.decorate(0)=fuzzinator.call.AdaptiveTimeoutDecorator

[sut.foo.call]
command=./bin/foo {test}
cwd=/home/alice/foo
timeout=3
class fuzzinator.call.AnonymizeDecorator(*, old_text, new_text=None, properties=None, **kwargs)

Decorator for SUT calls to anonymize issue properties.

Mandatory parameter of the decorator:

  • old_text: text to replace in issue properties.

Optional parameters of the decorator:

  • new_text: text to replace ‘old_text’ with (empty string by default).

  • properties: array of properties to anonymize (anonymize all properties by default).

Example configuration snippet:

[sut.foo]
call=fuzzinator.call.StdinSubprocessCall
call.decorate(0)=fuzzinator.call.AnonymizeDecorator

[sut.foo.call]
command=/home/alice/foo/bin/foo -

[sut.foo.call.decorate(0)]
old_text=/home/alice/foo
new_text=FOO_ROOT
properties=["stdout", "stderr"]
class fuzzinator.call.Call

Abstract base class to represent how a software-under-test (SUT) can be called/executed.

__call__(*, test, **kwargs)

Call SUT with test as input. Return a dictionary object if the input triggered an issue in the SUT, or a value considered false otherwise. (NonIssue, a dictionary that always evaluates to false, can be returned to communicate details about an input that did not trigger an issue.) The returned issue dictionary (if any) should contain an 'id' key that equals for issues that are not considered unique.

Raises NotImplementedError by default.

Parameters:

test (Any) – Input or test case, usually generated by a fuzzer. It is the responsibility of the configuration to ensure that the SUT can be called with the test case generated by the fuzzer.

Returns:

The result of the SUT call.

Return type:

dict[str, Any] or NonIssue or None

__enter__()

Set up steps before calling the SUT potentially multiple times.

No-op by default.

__exit__(*exc)

Tear down steps after calling the SUT.

No-op by default.

class fuzzinator.call.CallDecorator

Base class for SUT call (i.e., Call) decorators.

__call__(call_class)

Return a decorated version of call_class. Create a subclass of call_class that transfers control to init, enter, exit, or call when its __init__, __enter__, __exit__, or __call__ methods are invoked.

Parameters:

call_class – The SUT call class to decorate.

Returns:

The decorated version of the SUT call class.

call(cls, obj, *, test, **kwargs)

Call obj of type cls. The default operation is to call the __call__ method of the original version of the SUT call class and return its result.

Subclasses of CallDecorator may override this method if customization of calling the SUT is needed. Usually, the overridden method has to call the original __call__ at some point, which can be performed either by super().call(cls, obj, test=test, **kwargs) (which will call this method, and then transitively the original __call__) or by super(cls, obj).__call__(test=test, **kwargs) (which calls the original __call__ directly).

Parameters:

  • cls – The decorated version of the SUT call class, as returned by __call__.

  • obj – The SUT call instance to invoke.

  • test – Input or test case for the SUT call, as defined by Call.__call__.

Returns:

The result of the SUT call, as defined by Call.__call__.

enter(cls, obj)

Enter the context managed by obj of type cls. The default operation is to call the __enter__ method of the original version of the SUT call class and return its result.

Subclasses of CallDecorator may override this method if customization of entering the context is needed. Usually, the overridden method has to call the original __enter__ at some point, which can be performed either by super().enter(cls, obj) (which will call this method, and then transitively the original __enter__) or by super(cls, obj).__enter__() (which calls the original __enter__ directly).

Parameters:

  • cls – The decorated version of the SUT call class, as returned by __call__.

  • obj – The SUT call instance managing the context.

Returns:

A value as defined by the context management protocol (usually, obj).

exit(cls, obj, *exc)

Exit the context managed by obj of type cls. The default operation is to call the __exit__ method of the original version of the SUT call class and return its result.

Subclasses of CallDecorator may override this method if customization of exiting the context is needed. Usually, the overridden method has to call the original __exit__ at some point, which can be performed either by super().exit(cls, obj, *exc) (which will call this method, and then transitively the original __exit__) or by super(cls, obj).__exit__(*exc) (which calls the original __exit__ directly).

Parameters:

  • cls – The decorated version of the SUT call class, as returned by __call__.

  • obj – The SUT call instance managing the context.

  • exc – The exception that caused the context to be exited (if any), as defined by the context management protocol.

Returns:

Whether to suppress the exception in exc, as defined by the context management protocol.

init(cls, obj, **kwargs)

Initialize obj of type cls. The default operation is to call the __init__ method of the original version of the SUT call class.

Subclasses of CallDecorator may override this method if customization of the initialization is needed. Usually, the overridden method has to call the original __init__ at some point, which can be performed either by super().init(cls, obj, **kwargs) (which will call this method, and then transitively the original __init__) or by super(cls, obj).__init__(**kwargs) (which calls the original __init__ directly).

Parameters:

  • cls – The decorated version of the SUT call class, as returned by __call__.

  • obj – The SUT call instance to initialize.

class fuzzinator.call.ExitCodeFilter(*, exit_codes, invert=False, **kwargs)

Decorator filter for SUT calls that return issues with 'exit_code' property.

Mandatory parameter of the decorator:

  • exit_codes: if issue['exit_code'] is not in the array of exit_codes, the issue is filtered out; this behavior can be inverted by setting the 'invert' property to True, thus keeping issues with exit code not listed in the array.

Optional parameter of the decorator:

  • invert: if it’s true then exit code filtering mechanism is inverted (boolean value, False by default).

The issues that are not filtered out are not changed in any way.

Example configuration snippet:

[sut.foo]
call=fuzzinator.call.StdinSubprocessCall
call.decorate(0)=fuzzinator.call.ExitCodeFilter

[sut.foo.call]
command=/home/alice/foo/bin/foo -

[sut.foo.call.decorate(0)]
exit_codes=[139]
invert=false
class fuzzinator.call.FileReaderDecorator(*, cleanup=False, **kwargs)

Decorator for SUTs that take input as a file path: saves the content of the failing test case and optionally removes the file.

Moreover, the issue (if any) is also extended with the new 'filename' property containing the name of the test case (as received in the test argument).

Optional parameter of the decorator:

  • cleanup: Boolean to enable the removal of the loaded file (default: False).

Example configuration snippet:

[sut.foo]
call=fuzzinator.call.SubprocessCall
call.decorate(0)=fuzzinator.call.FileReaderDecorator

[sut.foo.call]
# assuming that foo takes one file as input specified on command line
command=/home/alice/foo/bin/foo {test}

[sut.foo.decorate(0)]
cleanup=True
class fuzzinator.call.FileWriterDecorator(*, filename, work_dir, **kwargs)

Decorator for SUTs that take input from a file: writes the test input to a temporary file (relative to an implicit temporary working directory) and replaces the test input with the name of that file.

Mandatory parameter of the decorator:

  • filename: name pattern for the temporary file, which may contain the substring {uid} as a placeholder for a unique string (replaced by the decorator).

The issue returned by the decorated SUT (if any) is extended with the new 'filename' property containing the name of the generated file (although the file itself is removed).

Example configuration snippet:

[sut.foo]
call=fuzzinator.call.SubprocessCall
call.decorate(0)=fuzzinator.call.FileWriterDecorator

[sut.foo.call]
# assuming that foo takes one file as input specified on command line
command=/home/alice/foo/bin/foo {test}

[sut.foo.call.decorate(0)]
filename=test-{uid}.txt
class fuzzinator.call.GdbBacktraceDecorator(*, command, cwd=None, env=None, encoding=None, **kwargs)

Decorator for subprocess-based SUT calls with file input to extend issues with 'backtrace' property.

Mandatory parameter of the decorator:

  • command: string to pass to GDB as a command to run (all occurrences of {test} in the string are replaced by the actual name of the test file).

Optional parameters of the decorator:

  • cwd: if not None, change working directory before GDB/command invocation.

  • env: if not None, a dictionary of variable names-values to update the environment with.

  • encoding: stdout and stderr encoding (default: autodetect).

The new 'backtrace' issue property will contain the result of GDB’s bt command after the halt of the SUT.

Example configuration snippet:

[sut.foo]
call=fuzzinator.call.SubprocessCall
call.decorate(0)=fuzzinator.call.GdbBacktraceDecorator

[sut.foo.call]
# assuming that {test} is something that can be interpreted by foo as
# command line argument
command=./bin/foo {test}
cwd=/home/alice/foo
env={"BAR": "1"}

[sut.foo.call.decorate(0)]
command=${sut.foo.call:command}
cwd=${sut.foo.call:cwd}
env={"BAR": "1", "BAZ": "1"}
class fuzzinator.call.LldbBacktraceDecorator(*, command, cwd=None, env=None, timeout=None, encoding=None, **kwargs)

Decorator for subprocess-based SUT calls with file input to extend issues with 'backtrace' property.

Mandatory parameter of the decorator:

  • command: string to pass to Lldb as a command to run (all occurrences of {test} in the string are replaced by the actual name of the test file).

Optional parameters of the decorator:

  • cwd: if not None, change working directory before Lldb/command invocation.

  • env: if not None, a dictionary of variable names-values to update the environment with.

  • timeout: timeout (in seconds) to wait between two lldb commands (integer number, 1 by default).

  • encoding: stdout and stderr encoding (default: autodetect).

The new 'backtrace' issue property will contain the result of Lldb’s bt command after the halt of the SUT.

Example configuration snippet:

[sut.foo]
call=fuzzinator.call.SubprocessCall
call.decorate(0)=fuzzinator.call.LldbBacktraceDecorator

[sut.foo.call]
# assuming that {test} is something that can be interpreted by foo as
# command line argument
command=./bin/foo {test}
cwd=/home/alice/foo
env={"BAR": "1"}

[sut.foo.call.decorate(0)]
command=${sut.foo.call:command}
cwd=${sut.foo.call:cwd}
env={"BAR": "1", "BAZ": "1"}
class fuzzinator.call.NonIssue

Wrapper class for issue dictionaries to mark them as non-issue.

The NonIssue class is a custom dictionary with a truth value forced to False. With this change, it’s capable to express that an issue dictionary doesn’t encode a failure, without emptying it or converting to None. The content of a NonIssue can be taken as a SUT response to a given test case and can be useful in feedback-driven fuzzing scenarios.

class fuzzinator.call.PlatformInfoDecorator(**kwargs)

Decorator for SUT calls to extend issues with 'platform' and 'node' properties.

The new 'platform' issue property will contain the result of Python’s platform.platform and the 'node' property will contain the result of platform.node.

Example configuration snippet:

[sut.foo]
#call=...
call.decorate(0)=fuzzinator.call.PlatformInfoDecorator
class fuzzinator.call.RegexAutomaton(instructions, existing_fields=None)

Auxiliary class to recognize patterns in textual data and process them with user-defined automaton-like instructions.

Two kinds of instructions can be assigned to every pattern which are encoded as a prefix to the regex pattern in the form of: m<inst1><inst2> /<pattern>/. The first instruction defines what to do with the groups of a matching pattern:

  • s: Save every group into the issue dictionary even if it exists.

  • a: Add only new fields to the issue dictionary.

  • c: Clear the whole issue dictionary.

  • n: Do not add any groups to the issue dictionary.

The second instruction defines how to continue the processing of the input after a match:

  • c: Continue the processing of the current line with the next pattern.

  • s: Stop the processing of the current line and continue with the next line.

  • t: Terminate the processing of the whole input and return with the current version of the issue dictionary.

If a pattern lacks the instructions and the slashes then it is interpreted as an msc /<pattern>/.

class fuzzinator.call.RegexAutomatonFilter(**kwargs)

Decorator filter for SUT calls to recognise patterns in the returned issue dictionaries and process them according to user-defined automaton-like instructions.

Optional parameters of the decorator:

  • key: array of patterns to match against the lines of issue[key] (note that ‘key’ can be arbitrary, and multiple different keys can be given to the decorator). The patterns and instructions are interpreted as defined in fuzzinator.call.RegexAutomaton.

Example configuration snippet:

[sut.foo]
call=fuzzinator.call.StdinSubprocessCall
call.decorate(0)=fuzzinator.call.RegexAutomatonFilter

[sut.foo.call]
command=/home/alice/foo/bin/foo -

[sut.foo.call.decorate(0)]
stderr=["mss /(?P<file>[^:]+):(?P<line>[0-9]+): (?P<func>[^:]+): (?P<msg>Assertion `.*' failed)/"]
backtrace=["mas /#[0-9]+ +0x[0-9a-f]+ in (?P<path>[^ ]+) .*? at (?P<file>[^:]+):(?P<line>[0-9]+)/"]
class fuzzinator.call.RegexFilter(**kwargs)

Decorator filter for SUT calls to recognise patterns in the returned issue dictionaries.

Optional parameters of the decorator:

  • key: array of patterns to match against issue[key] (note that ‘key’ can be arbitrary, and multiple different keys can be given to the decorator).

If none of the patterns matches on any of the fields, the issue is filtered out. The issues that are not filtered out are extended with keys-values from the named groups of the matching regex pattern.

Example configuration snippet:

[sut.foo]
call=fuzzinator.call.StdinSubprocessCall
call.decorate(0)=fuzzinator.call.RegexFilter

[sut.foo.call]
command=/home/alice/foo/bin/foo -

[sut.foo.call.decorate(0)]
stderr=["(?P<file>[^:]+):(?P<line>[0-9]+): (?P<func>[^:]+): (?P<msg>Assertion `.*' failed)"]
backtrace=["#[0-9]+ +0x[0-9a-f]+ in (?P<path>[^ ]+) .*? at (?P<file>[^:]+):(?P<line>[0-9]+)"]
class fuzzinator.call.SanitizedStreamMonitoredSubprocessCall(**kwargs)

SanitizedStreamMonitoredSubprocessCall is a specialized fuzzinator.call.StreamMonitoredSubprocessCall that contains hard-wired regexes for ASAN and UBSAN error messages. Otherwise, it accepts parameters the same way as fuzzinator.call.StreamMonitoredSubprocessCall. If custom end_patterns are specified, they take precedence over the predefined patterns.

Note

Not available on platforms without fcntl support (e.g., Windows).

class fuzzinator.call.SanitizerAnalyzerDecorator(**kwargs)

Decorator for SUT calls to beautify the sanitizer-specific error messages and extend issues with the 'security' property which describes the severity of the issue. This decorator has to be used with fuzzinator.call.SanitizerAutomatonFilter.

Example configuration snippet:

[sut.foo]
#call=...
call.decorate(0)=fuzzinator.call.SanitizerAutomatonFilter
call.decorate(1)=fuzzinator.call.SanitizerAnalyzerDecorator
class fuzzinator.call.SanitizerAutomatonFilter(sanitized_properties=None, **kwargs)

SanitizerAutomatonFilter is a specialized fuzzinator.call.RegexAutomatonFilter that contains hard-wired regexes for ASAN and UBSAN error messages. The streams where these regexes are looked for can be defined with the sanitized_properties parameter. Otherwise, it accepts optional parameters the same way as fuzzinator.call.RegexAutomatonFilter. If they are specified, they take precedence over the predefined patterns. If an ASAN or UBSAN specific pattern is recognized, then issue dictionary will be extended with the error_type field.

Optional parameter of the decorator:

  • sanitized_properties: array of field names in the issue dictionary where the patterns are looked for (default: ["stderr"]).

Example configuration snippet:

[sut.foo]
call=fuzzinator.call.StdinSubprocessCall
call.decorate(0)=fuzzinator.call.SanitizerAutomatonFilter

[sut.foo.call]
command=/home/alice/foo/bin/foo -

[sut.foo.call.decorate(0)]
stdout=["mss /(?P<file>[^:]+):(?P<line>[0-9]+): (?P<func>[^:]+): (?P<msg>Assertion `.*' failed)/"]
backtrace=["mas /#[0-9]+ +0x[0-9a-f]+ in (?P<path>[^ ]+) .*? at (?P<file>[^:]+):(?P<line>[0-9]+)/"]
sanitized_properties=["stdout"]
class fuzzinator.call.StdinSubprocessCall(*, command, cwd=None, env=None, no_exit_code=None, timeout=None, encoding=None, **kwargs)

Subprocess invocation-based call of a SUT that takes a test input on its stdin stream.

Mandatory parameter of the SUT call:

  • command: string to pass to the child shell as a command to run.

Optional parameters of the SUT call:

  • cwd: if not None, change working directory before the command invocation.

  • env: if not None, a dictionary of variable names-values to update the environment with.

  • no_exit_code: makes possible to force issue creation regardless of the exit code.

  • timeout: run subprocess with timeout.

  • encoding: stdout and stderr encoding (default: autodetect).

Result of the SUT call:

  • If the child process exits with 0 exit code, no issue is returned.

  • Otherwise, an issue with 'exit_code', 'stdout', 'stderr' and 'time' properties is returned.

Example configuration snippet:

[sut.foo]
call=fuzzinator.call.StdinSubprocessCall

[sut.foo.call]
command=./bin/foo -
cwd=/home/alice/foo
env={"BAR": "1"}
class fuzzinator.call.StreamMonitoredSubprocessCall(*, command, cwd=None, env=None, end_patterns=None, timeout=None, encoding=None, **kwargs)

Subprocess invocation-based call of a SUT that takes test input on its command line. The main difference from fuzzinator.call.SubprocessCall is that it continuously monitors the stdout and stderr streams of the SUT and forces it to terminate if some predefined patterns are appearing.

Mandatory parameter of the SUT call:

  • command: string to pass to the child shell as a command to run (all occurrences of {test} in the string are replaced by the actual test input).

Optional parameters of the SUT call:

  • cwd: if not None, change working directory before the command invocation.

  • env: if not None, a dictionary of variable names-values to update the environment with.

  • end_patterns: array of patterns to match against the lines of stdout and stderr streams. The patterns and instructions are interpreted as defined in fuzzinator.call.RegexAutomaton.

  • timeout: run subprocess with timeout.

  • encoding: stdout and stderr encoding (default: autodetect).

Result of the SUT call:

  • If processing stdout and stderr with end_patterns doesn’t produce any result, no issue is returned.

  • Otherwise, an issue with keys from the matching patterns of end_pattern extended with the 'exit_code', 'stdout', 'stderr' and 'time' properties is returned.

Example configuration snippet:

[sut.foo]
call=fuzzinator.call.StreamMonitoredSubprocessCall

[sut.foo.call]
# assuming that {test} is something that can be interpreted by foo as
# command line argument
command=./bin/foo {test}
cwd=/home/alice/foo
env={"BAR": "1"}
end_patterns=["mss /(?P<file>[^:]+):(?P<line>[0-9]+): (?P<func>[^:]+): (?P<msg>Assertion `.*' failed)/"]
timeout=30

Note

Not available on platforms without fcntl support (e.g., Windows).

class fuzzinator.call.SubprocessCall(*, command, cwd=None, env=None, no_exit_code=None, timeout=None, encoding=None, **kwargs)

Subprocess invocation-based call of a SUT that takes test input on its command line. (See fuzzinator.call.FileWriterDecorator for SUTs that take input from a file.)

Mandatory parameter of the SUT call:

  • command: string to pass to the child shell as a command to run (all occurrences of {test} in the string are replaced by the actual test input).

Optional parameters of the SUT call:

  • cwd: if not None, change working directory before the command invocation.

  • env: if not None, a dictionary of variable names-values to update the environment with.

  • no_exit_code: makes possible to force issue creation regardless of the exit code.

  • timeout: run subprocess with timeout.

  • encoding: stdout and stderr encoding (default: autodetect).

Result of the SUT call:

  • If the child process exits with 0 exit code, no issue is returned.

  • Otherwise, an issue with 'exit_code', 'stdout', 'stderr' and 'time' properties is returned.

Example configuration snippet:

[sut.foo]
call=fuzzinator.call.SubprocessCall

[sut.foo.call]
# assuming that {test} is something that can be interpreted by foo as
# command line argument
command=./bin/foo {test}
cwd=/home/alice/foo
env={"BAR": "1"}
class fuzzinator.call.SubprocessPropertyDecorator(*, property, command, cwd=None, env=None, timeout=None, encoding=None, **kwargs)

Decorator for SUT calls to extend issues with an arbitrary property where the value is the output of a shell subprocess.

Mandatory parameters of the decorator:

  • property: name of the property to extend the issue with.

  • command: string to pass to the child shell as a command to run.

Optional parameters of the decorator:

  • cwd: if not None, change working directory before the command invocation.

  • env: if not None, a dictionary of variable names-values to update the environment with.

  • timeout: run subprocess with timeout.

  • encoding: stdout and stderr encoding (default: autodetect).

Example configuration snippet:

[sut.foo]
call=fuzzinator.call.StdinSubprocessCall
call.decorate(0)=fuzzinator.call.SubprocessPropertyDecorator

[sut.foo.call]
command=./bin/foo -
cwd=/home/alice/foo

[sut.foo.call.decorate(0)]
property=version
command=git rev-parse --short HEAD
cwd=${sut.foo.call:cwd}
env={"GIT_FLUSH": "1"}
class fuzzinator.call.TestRunnerSubprocessCall(*, command, cwd=None, env=None, end_texts=None, init_wait=None, timeout_per_test=None, encoding=None, **kwargs)

Note

Not available on platforms without fcntl support (e.g., Windows).

class fuzzinator.call.UniqueIdDecorator(*, properties, **kwargs)

Decorator for SUT calls to extend issues with 'id' property.

Mandatory parameter of the decorator:

  • properties: array of issue property names, which are concatenated (separated by a space) to form the new 'id' property.

Example configuration snippet:

[sut.foo]
call=fuzzinator.call.StdinSubprocessCall
call.decorate(0)=fuzzinator.call.RegexFilter
call.decorate(1)=fuzzinator.call.UniqueIdDecorator

[sut.foo.call]
command=/home/alice/foo/bin/foo -

[sut.foo.call.decorate(0)]
stderr=[": (?P<file>[^:]+):(?P<line>[0-9]+): (?P<func>[^:]+): (?P<msg>Assertion `.*' failed)"]

[sut.foo.call.decorate(1)]
properties=["msg", "file", "func"]